28.08.2019 
In the previous two blogs in this series I showed how solving an apparently simple problem about loading a lot of data into RAM using mmap() also turned out to require a solution that improved CPU use across cores. In this blog, I’ll show how we dealt with the bottleneck problems that ensued, and finally, […]
Blog Archives
Little Trouble in Big Data – Part 2 improving CPU utilization across cores
In Part 1 of this series we solved the problem of using mmap() to load large set of data into RAM all at once. The catch was that it still took too long. In this blog, I describe how we solve this.
Little Trouble in Big Data – Part 1 how to use mmap() to load a large data set into RAM
We got a call from a bioinformatics group at a European university. The problem appeared simple, how to usemmap() to load a large data set into RAM all at once. OK, no problem I thought...
Making the Most of your Memory with mmap
Today’s complex applications can soak up megabytes, a memory management strategy is crucial for RAM intensive applications like image/video processing, massive databases, and machine learning.