Working with databases can be a pain when you are designing a low-latency flow in software.
I am not a database expert, yet I have achieved good results with following ideas.

1. In-memory Database instead of RDBMS
Redis, MongoDb and many others free and paid solutions are available. Every solution addresses the same problem in a different way.
Experiment first if the run-time numbers on large insertions and searches meet your criteria.
And the database organizes the data in the way you want it. Evaluate well at design phase.

2. Configuration Parameters
Every RDBMS has some configuration parameters that can be exploited per your needs.
These days RAM is cheaper. So having a big cache size helps.
The other day, I reduced the insertion time in SQLLite by 20 times just by enabling PRAGMA synchronous=OFF
So, read the documentation well.

3. Separate thread
Mostly, INSERT into database are the slowest transactions.
Separate thread or process can be spawned to provide INSERT apis.

4. Design your tables and queries well
Use Indexes with Tables.
Design complex SQL queries properly to save run-time. You can run analyze tools available with RDBMS package for the purpose.

5. Avoid databases
Databases are required only when data is very large, relations are required between tables and you need complex queries on top of them.
In-memory data-structures are useful for in-flow systems.
mmap() may be useful for small files.
http://pubs.opengroup.org/onlinepubs/009695399/functions/mmap.html
http://www.linuxquestions.org/questions/programming-9/mmap-is-not-faster-than-read-write-931799/

More Ideas on reducing latency

Tagged with →  
Share →

3 Responses to Reducing Database Latency

  1. Good article and right to the point. I am not sure if this is in fact the best place to ask but do you folks have any ideea where to hire some professional writers? Thanks in advance :)

  2. I simply wanted to type a brief remark in order to thank you for the great concepts you are placing at this site. My rather long internet research has at the end of the day been honored with beneficial know-how to go over with my good friends. I ‘d believe that many of us website visitors actually are quite fortunate to be in a useful place with so many marvellous individuals with very helpful tips and hints.

  3. very good!!:)thanks very much:)..

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>