Greetings my fellow OHDSIans! I am just getting started with OHDSI, CMS SynPUF data and am only slightly more advanced using Python and Linux on AWS. I could really use some tips on how to download CMS SynPUF files to an S3 bucket in AWS.
When I run the get_synpuf_files.py scrip from the OHDSI ETL-CMS-master package in my AWS Cloud9 environment, it works great, but my local directory quickly runs out of space. So I had the idea of mounting an S3 bucket to the local directory named “cms-synpuf-files”.
Using the s3fs utility, I successfully mounted an S3 bucket to the local directory (note: I used this procedure to mount the directory: https://cloudkul.com/blog/mounting-s3-bucket-linux-ec2-instance/) )
However, when running the get_synpuf_files.py script, I get input/output and permissions types of errors on the “cms-synpuf-files” folder. Errors include
- “PermissionError: [Errno 13] Permission denied: ‘cms-synpuf-files/DE_1/DE1_0_2008_Beneficiary_Summary_File_Sample_1.zip’ “
- When typing “ls” in the Bash prompt: “reading directory ‘cms-synpuf-files’: Input/output error”
- when typing “ls” in the parent directory, I get this record for the cms-synpuf-files folder:
“drwxrwxr-x. 1 1001 ec2-user 4096 Jan 1 1970 cms-synpuf-files”
Has anyone had any luck downloading CMS SynPUF files to an AWS S3 bucket and can help out a OHDSI newbie?
Thanks! Travis