Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

One recent headache I had, I had to download JSON files loaded since yesterday's (about 15k) using python. The issue is that the bucket had 20+ million files. The naming convention (folders) wasn't perfect unfortunately. There wasn't a way to do this filtering on the server side. I had to download the file list with the metadata and then make calls to get each file individually based on metadata date.

Not a smooth process.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: