XUtils

aws-sdk-pandas

Pandas on AWS.


Table of contents

Storing data on Data Lake

wr.s3.to_parquet(

df=df,
path="s3://bucket/dataset/",
dataset=True,
database="my_db",
table="my_table"

)

Retrieving the data directly from Amazon S3

df = wr.s3.read_parquet(“s3://bucket/dataset/”, dataset=True)

Retrieving the data from Amazon Athena

df = wr.athena.read_sql_query(“SELECT * FROM my_table”, database=“my_db”)

Get a Redshift connection from Glue Catalog and retrieving data from Redshift Spectrum

con = wr.redshift.connect(“my-glue-connection”) df = wr.redshift.read_sql_query(“SELECT * FROM external_schema.my_table”, con=con) con.close()

Amazon Timestream Write

df = pd.DataFrame({

"time": [datetime.now(), datetime.now()],   
"my_dimension": ["foo", "boo"],
"measure": [1.0, 1.1],

}) rejected_records = wr.timestream.write(df,

database="sampleDB",
table="sampleTable",
time_col="time",
measure_col="measure",
dimensions_cols=["my_dimension"],

)

Amazon Timestream Query

wr.timestream.query(“”” SELECT time, measure_value::double, my_dimension FROM “sampleDB”.“sampleTable” ORDER BY time DESC LIMIT 3 “”“)


## Getting Help

The best way to interact with our team is through GitHub. You can open an [issue](https://github.com/aws/aws-sdk-pandas/issues/new/choose) and choose from one of our templates for bug reports, feature requests...
You may also find help on these community resources:
* The #aws-sdk-pandas Slack [channel](https://join.slack.com/t/aws-sdk-pandas/shared_invite/zt-sxdx38sl-E0coRfAds8WdpxXD2Nzfrg)
* Ask a question on [Stack Overflow](https://stackoverflow.com/questions/tagged/awswrangler)
  and tag it with `awswrangler`
* [Runbook](https://github.com/aws/aws-sdk-pandas/discussions/1815) for AWS SDK for pandas with Ray

## Logging

Enabling internal logging examples:

```py3
import logging
logging.basicConfig(level=logging.INFO, format="[%(name)s][%(funcName)s] %(message)s")
logging.getLogger("awswrangler").setLevel(logging.DEBUG)
logging.getLogger("botocore.credentials").setLevel(logging.CRITICAL)

Into AWS lambda:

import logging
logging.getLogger("awswrangler").setLevel(logging.DEBUG)

Articles

  • coming soon...