Why use data virtualization?

Damn large amounts of data

The first use case for DV that jumps into your head is to virtualize data sources with gigantic amounts of entries. What is the maximum amount we can access via IList? Well, the Count property is of type int, So 2,147,483,647 (int.MaxValue) is exactly the amount of entries we can virtaulize at most. In a tab of the sample app of this project all positive integers from 0 to 2,147,483,646 (int.MaxValue - 1) are virtualized. With such a huge amount, even with 32 bits per entry, you have to think about the load on your memory. 2,147,483,648 times 32 bits are 68,719,476,736 bits, divided by 8 bits it is 8,589,934,592 bytes. So about 8 GB. If the list was not virtualized, we would need 8 GB of memory just to work with a list of all positive numbers.

Entries with a high workload

Okay, okay, when do you work with such gigantic quantities? Yeah, that's a good point. But say we want a list of book entries. Each entry should have a relatively high resolution picture of the front page. Even with a few thousand entries, it becomes uncomfortable for the memory. Especially you have to consider that in an application probably only a handful of entries can be displayed at once. The remaining entries would be uselessly loaded into memory beforehand. Only when you scroll to these entries they would serve a purpose, but probably not all entries would be viewed during a session with the application anyway.

Let's stick to the example with the title pages of books. Are we talking about milliseconds or seconds per image loaded? Are the images all available locally or do they have to be downloaded from the Internet? With DV it makes little difference, because if only the content that is currently displayed is loaded, the time to load and the memory usage is always kept as low as possible.

Entries with high performance requirements

Another use case concerns the runtime required for each list entry. Let's say the entries are modeled by a class that lets its objects all listen for a central event, and the processing of the event takes a few milliseconds per object. With thousands of entries we would be in the range of seconds altogether. That means we are not far from UI-freezes. Now one could think that the class is badly modeled if it allows UI-freezes. If you consider the possibilities of DV, you will soon realize that bad modeling means having to instantiate objects that bring no benefit but only performance overloads. After all, a basic principle of programming says that one should not optimize eagerly. DV gives you more possibilities to model classes of entries.