How I optimized Firestore read/write by 2000 times

If you use Firebase and Firestore for long enough, you gonna realize that the read/write times are the biggest impact on your invoice every month.
I have this problem for my pet project
Basically, I have about 65k quotes that gonna send to the browser for running offline mode. The app is doesn't have any users, except me, and let's think about this. If 1 user takes 65k read every day, what if I have 50 users?
65k * 50 * 30 days = 97 500k
notion image
Which cost me about 17.5$ month
That's a lots for just 50 users and a pet project, if I use Cloud computing, it is more affordable solutions.

How can I reduce read/wite to 15 times

When I learn NEAR Protocal I found that they used sharding - which is an approach to optimization that comes from database.
notion image
By changing the structure of #FireStore from each item into a new doc, we can combine multiple items into a doc and saving thousand of reading/writing time.
notion image
I store about 200 quotes in each documents, you can think that each my document is the same as pagination query.
So that instead of costing reading 2000 records, It just cost 1 read. 2000 times decreasing!

What does it cost?

notion image
Well, the most cost is, you need to programing to get your desire result, no more query, filter, or order supported by the SDK, you need to do it by yourself
But believe me, you gonna do that by yourself if you use Firestore, it's suck 🙂
notion image
Firestore looks awesome when we first start and then it's suck when you found out those limitations
You can't even count how many documents you have in a collection

How should I split my documents?

It depends, this is the hardest question when you implement sharding. For my application, I just separate each shard by 200 quotes. You have more about your application context, so you're the best to answer on How to separate your documents
After having a strategy on separating your document, you also need some effort when inserting/updating your document. Which is hard or easy is based on your strategy

When should I use sharding for Firestore?

You are happy on coding to query documents
You have a strategy on sharding that can answers
  • How can I separate my document?
  • By your separate strategy, how can I find the document I desire
  • How can I insert/update new documents? Which shard should I put it in
If you have a clear answer to those questions? Get ready on saving your billing by 2000 times

Loading Comments...

Follow me @thanhledev

Xin lỗi các bạn vì thời gian qua mình không dành thời gian viết nhiều. Dạo này mình khá bận cho dự án Check it out 🥳