Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Parts may have been , but it was ~2B rows at ~ 1K/row in PG, with a couple of embedded json fields (which were probably compressed, but ... that's coming from a bad starting point anyway).

It got converted to a table with ~25 Float32 columns + a couple of key columns.



Yes it's an 5-apples vs 1-orange comparison, especially the dynamic json fields where keys are repeated.


Fixing the PG schema would have saved me about 2/3 of the space. (from a small test run) Good, but not enough to move the needle.

Queries on the PG setup here are back to the bad old days of being limited by bulk disk performance.


> Good, but not enough to move the needle

You could reduce the gap by running PG on compressed filesystem, so it would be more apples to apples comparison.


It would be 1-apple to 1-orange comparison. If the columnar store has an "ok" architecture, it should provide 10x+ faster queries.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: