Direct Lake Internals: Transcoding, Framing, Residency, and Eviction
- Vojtěch Šíma
- Jul 22
- 21 min read
Updated: Jul 26
TL;DR: Direct Lake is a semantic model storage mode that combines "DirectQuery-style" data freshness (not literally) with near-Import performance. It is available only in Microsoft Fabric, using Delta tables stored in OneLake, and can read data either directly from Delta tables or through the SQL analytics endpoint. After the model has warmed up, Direct Lake typically delivers performance comparable to Import mode, though it consumes more capacity units (CUs). Warming effectively caches individual columns in memory, enabling Import-like query speed. However, this warming can be slow, so you should play around that and prewarm it for users. Direct Lake works best with well-optimized, properly maintained delta tables and should follow the same data-modeling principles as a traditional Import semantic model.
Disclaimer: If not said otherwise, DirectLake benchmarks are done on DirectLake on OneLake. Every semantic model also gets data from single Lakehouse, regardless of storagemode.
What is Direct Lake?
Direct Lake is a Power BI semantic model table storage option. This mode expands the already existing options such as Import and DirectQuery.
For a quick picture in your head, imagine Direct Lake as a combination of Import mode for relatively quick responses and DirectQuery for the lack of a traditional semantic model refresh.
However, that does not tell the whole story.
Direct Lake works together with Microsoft Fabric and its OneLake storage, or more precisely, with the Delta tables stored in it. Instead of importing and storing another full copy of the data inside the semantic model, Direct Lake loads the required columns directly from the Delta tables into memory when they are needed.
Once loaded, those columns are stored in the same VertiPaq structures used by Import mode and can be served to the report from memory.
So, in short, and I will explain this in more detail later, you pay for the initial warming of the cache. Then, as long as the required data stays warm and does not get evicted, you should get performance similar to Import mode.
In theory.
Direct Lake on SQL
Direct Lake currently has two flavors.
Direct Lake on SQL is mainly useful when you need SQL analytics endpoint objects or SQL-based security. It uses the SQL analytics endpoint of a Lakehouse or Warehouse for schema discovery and permission checks. For compatible physical tables, it still loads the backing Delta data from OneLake.
If you use SQL views or have SQL RLS, it may/will fall back to DirectQuery, and the performance will be worse.
You also cannot add additional data sources to the semantic model, other than other Direct Lake on SQL tables from the same Fabric data source.
For me, this is the less interesting mode. Unless stated otherwise, the rest of the article focuses on Direct Lake on OneLake.
Direct Lake on OneLake
The OneLake flavor is a bit more interesting. It connects directly to the Delta tables and eliminates the SQL analytics endpoint sync. You are also using a different connector in the background, and you are basically reading the files directly.
There is naturally no fallback to DirectQuery, as there is no SQL to run. You can also have additional tables using Import mode.
Unless stated otherwise, the article covers the behavior of Direct Lake on OneLake.
How to create a semantic model with Direct Lake
I find it easiest to build it through the Fabric/Power BI service, either through a new Item in your workspace or through New semantic model in your Lakehouse/Warehouse. You can also do it in Power BI Desktop, but it feels easier to start in the web and then edit it in Desktop.
With that setup, you can't choose the flavor of Direct Lake. If you want to lock it in from the get-go, go to your Warehouse or Lakehouse and its SQL Analytics Endpoint, and click New semantic model. Then you get the Storage Mode option as well.
Here you define the name, the workspace where it will live, the Storage Mode, and finally the tables you want to use. Once you do that, you'll be transferred to the Semantic Model edit/view window. If you want to edit it, you can choose whether to do so in the web or in Desktop, and from that point it's the standard way.
One thing I noticed: if you aren't yet logged in to Power BI Desktop, it may throw an error (Cannot access a disposed object. Object name: 'MainForm'.). Opening empty Power BI Desktop, logging in, and then trying again fixes it.
Since a Lakehouse is (or will be) schema-enabled by default, make sure you save your tables with the proper /schema/table path when building. If you don't specify the schema in a schema-enabled Lakehouse, it'll throw errors when you try to build the model. It'll be something like this: "We cannot refresh this semantic model because this semantic model uses a default data connection without explicit connection credentials. Please replace the default data connection in the semantic model settings with an explicit cloud or gateway data connection." The error can point to other causes too, but this is one of them.
How to validate Direct Lake's storage mode
The easiest way to validate what kind of storage you have is to go to Power BI Desktop, edit the semantic model, open TMDL, and when you browse the model, check out Expressions.
Then, just from the name DirectLake - default_lakehouse (that would be the name of your lakehouse, for example), you can see this is OneLake. If you want to inspect the code, open it in TMDL:
This is a clear sign of using the AzureStorage.DataLake() function, which means OneLake. You'll also see some other annotations like PBI_RemovedChildren. I'd advise not to touch this, as modifying it or editing new tables can cause some trouble.
If you see DatabaseQuery in the expression, that means Direct Lake on SQL:
To check whether you have a table as Direct Lake in general, again use TMDL.
You should see mode: directLake
If you were to copy the expression and try to create a Direct Lake model from scratch without using the UI, it won't work. At least not via Power BI Desktop. Even if you had the exact same queries and definitions. So always go through the UI first.
With OneLake storage mode, you can also add additional data, normally through Power Query, just as you're used to with Import or DirectQuery methods.
How does Direct Lake work?
Now, let's talk about how the whole thing works. My goal is to give you more information than what you can read in the official docs. The core definitions are taken from Microsoft documentation with my spin on them, but the further tests, findings, and benchmarks go beyond the docs.
Transcoding - Column Loading
This is one of the core operations of Direct Lake.
When you send a DAX query, typically when you open a Power BI report or directly query the semantic model, the engine checks all columns required to complete the request and loads the missing ones into memory.
This includes columns referenced by measures, filters, relationships, grouping, and basically anything else needed to fulfill the query.
This process is usually the slowest part of a cold Direct Lake query. However, it happens only when the column is requested for the first time or is no longer resident.
Repeating the same query after the initial load can match similar performance to Import, because once the data is loaded into memory, you are basically operating as you would with Import.
The transcoding speed depends primarily on the cardinality of the column and also on how the data is spread across files and row groups.
For example, a column with text GUIDs will take significantly more time than a low-cardinality numeric column.
The column stays resident in memory until it has a reason to move out. This can happen when:
you change the underlying data
you do not query the column for a long time
the engine needs to create space for more urgent columns
All of these scenarios will be explained further later.
Framing
Framing is lowkey just a refresh of the semantic model. More precisely, it defines which version of each Delta table Direct Lake should use. A Delta table is basically a list of Parquet files tracked in the Delta log. When you want to read a specific version, Delta uses the log to identify which files were active.
Framing creates this point-in-time reference for Direct Lake. Because of that, the framed version does not always have to match the latest Delta table version.
In terms of residency, framing dictates what data will eventually get loaded to the memory.
By default, the sync is automatic. When new data is written, Direct Lake usually moves to the latest Delta version within a short time (depends on your volumes, but could be within 10 seconds).
You can also disable automatic sync and control framing manually or programmatically. This is useful when you do not always need the latest version immediately, or when you want several table updates to become visible together.
Validation of the Framing
To compare the latest Delta version with the version currently used by Direct Lake, you need to read them from two different places.
You can get the latest Delta table version directly from the Delta log. This can be done with either Spark or a non-Spark library such as deltalake, which uses delta-rs.
With Spark, you can use the Spark Delta Lake API:
from delta.tables import DeltaTable
delta_version = (
DeltaTable.forName(spark, "dbo.dim_customer")
.history(1)
.select("version")
.first()["version"]
)Without Spark, you can use the deltalake library and open the table directly through its OneLake path:
from deltalake import DeltaTable
delta_table = DeltaTable(table_path)
delta_version = delta_table.version()The framed version is read separately from the semantic model:
EVALUATE
INFO.DELTATABLEMETADATASTORAGES()The important column is CurrentVersion, which shows the Delta version currently held by the semantic model.
I then disabled automatic sync, captured both versions and queried the semantic model. After that, I added new rows into every Delta table, checked the versions again, waited for a bit, and repeated the same query. Finally, I enabled automatic sync and measured everything once more.
from sempy_labs import directlake
directlake.set_autosync(
dataset="directlake_on_onelake",
workspace="direct_lake_consumption",
enable=False
)Stage | Autosync | Delta version | Framed version | New rows visible |
Before write | Disabled | 2 | 0 | 0 |
After write | Disabled | 3 | 0 | 0 |
After 30 seconds | Disabled | 3 | 0 | 0 |
After enabling autosync | Enabled | 3 | 3 | 9 |
Row groups
As mentioned earlier, a Delta table consists of Parquet files. A Parquet file stores data in a columnar format, but the file is also split horizontally into one or more row groups.
A row group is basically a batch of rows inside the Parquet file. Each row group contains data for all columns, but every column is stored separately in its own column chunk.
The number of row groups per file depends on how the file was written, including the file size, target row group size, writer engine, and batching behavior.
In our example, the main Parquet file contains 750,000 rows and only one row group. The three tiny append files contain only three rows each, but every file still has its own row group.
File type | Rows | Row groups | Columns | Column segments |
Main file | 750,000 | 1 | 7 | 7 |
Small 3 append files | 3 | 1 | 7 | 7 |
All 4 active files | 750,009 | 4 | 7 | 28 |
Optimized layout | 750,009 | 1 | 7 | 7 |
Column chunks and column segments
In Parquet, the column-level part of a row group is called a column chunk. When Direct Lake transcodes the data into VertiPaq, these column chunks become column segments inside the semantic model.
VertiPaq is the in-memory columnar storage engine used by Power BI to process DAX queries quickly. In Direct Lake, the required Parquet column chunks are transcoded into VertiPaq column segments when they are needed, rather than importing the whole model upfront.
So, if one row group has seven columns, it contains seven column chunks and creates seven column segments. If the Delta table has four row groups, every semantic model column has four segments.
These segments are what Direct Lake can load into memory, keep resident, or evict. If a query only needs a few columns, the engine only needs to load the segments for those columns instead of the entire table.
This is also why small files and too many row groups can become annoying. In our example, every three-row append file still creates seven new column chunks and seven new column segments, even though the file contains basically no data.
Each column chunk also stores metadata such as compression, encoding, null count, minimum, maximum, and compressed size.
Column | Rows in chunk | Compressed size | Min | Max |
customer_key | 750,000 | 3.6 MB | 1 | 750,000 |
customer_code | 750,000 | 3.8 MB | CUST-0000001 | CUST-0750000 |
customer_name | 750,000 | 3.8 MB | Customer 0000001 | Customer 0750000 |
segment | 750,000 | 191 KB | Consumer | Small Business |
country | 750,000 | 284 KB | Austria | Slovakia |
is_active | 750,000 | 87 KB | false | true |
This is a nice example of why columnar storage works so well. A low-cardinality column such as is_active is tiny, while string columns such as customer_name are much larger.
If a DAX query only needs segment and is_active, Direct Lake only loads the segments for those columns, not the whole Parquet file.
An unoptimized table for the
column customer_key with the appends could then look like this:
File type | File rows | Row group | Segment values | Compressed size | Min | Max |
Main file | 750,000 | 0 | 750,000 | 3,566,266 bytes | 1 | 750,000 |
Small file 1 | 3 | 0 | 3 | 85 bytes | 750001 | 750003 |
Small file 2 | 3 | 0 | 3 | 85 bytes | 750004 | 750006 |
Small file 3 | 3 | 0 | 3 | 57 bytes | 750007 | 750009 |
Local and global dictionaries
The last thing I want to cover before moving to residency is dictionaries.
Dictionaries are not only a VertiPaq thing. Both Parquet and VertiPaq can use dictionary encoding. Instead of storing the full value for every row, the dictionary stores each distinct value once and assigns it a numeric ID.
The dictionary size is therefore directly connected to column cardinality. A column such as is_active only needs two values, while a customer name or text GUID column might need hundreds of thousands or even millions.
In Parquet, dictionaries are local to individual column chunks. If a column is spread across four row groups, Direct Lake might need to process four separate local dictionaries for that column.
VertiPaq, on the other hand, uses one global dictionary for the whole semantic model column. In Import mode, this global dictionary is built during the refresh. In Direct Lake, it is built during transcoding by combining the local Parquet dictionaries and mapping their local IDs to the global IDs.
Source | Local dictionary | Stored local IDs | Global IDs |
Row group 1 | Consumer = 0, Small Business = 1 | 0, 0, 1 | 0, 0, 2 |
Row group 2 | Enterprise = 0, Consumer = 1 | 0, 1, 1 | 1, 0, 0 |
Row group 3 | Consumer = 0 | 0, 0, 0 | 0, 0, 0 |
Global VertiPaq dictionary | Consumer = 0, Enterprise = 1, Small Business = 2 |
This means that more row groups can also mean more local dictionaries that Direct Lake needs to process and merge. Yet another reason why creating thousands of tiny files and row groups is probably not the best idea.
Validation of Dictionaries
To validate the local dictionaries, you can read the parquet directly through Python.
parquet_file = parquet.ParquetFile(
parquet_file_path,
read_dictionary=["segment"]
)
dictionary_array = (
parquet_file
.read_row_group(0, columns=["segment"])
.column("segment")
.chunk(0)
)
dictionary_array.dictionary.to_pylist()
dictionary_array.indices.to_pylist()For the global dictionary, you can't get the nice pairing with index and actual values; you can only get some idea through:
EVALUATE
INFO.DICTIONARYSTORAGES()Pairing it with:
DISTINCT('dim_customer'[segment])Residency and temperature
Once Direct Lake transcodes a Parquet column chunk, the resulting VertiPaq column segment can become resident, meaning it is currently loaded in memory and can be reused by further queries. In short, the column is ready to serve you the data without any delay, apart from your own DAX and model.
The clearest way to check this is ISRESIDENT from:
INFO.STORAGETABLECOLUMNSEGMENTS()That loading step can dominate the visual response time. In one of my tests, the first query took 33.11 seconds while the required data was being loaded. Once the same working set was resident, the following executions completed in approximately one second.
Note, that the hunders of seconds were primarily for stress testing but also benchmark for how badly designed parquet layout affect transcoding.
Once the required structures are resident, further queries can reuse them directly. This is also why it is possible to preload parts of a Direct Lake model by executing queries against the columns and relationships that users are likely to need. I will come back to that later.
Each resident segment also has a temperature, which changes based on how recently and frequently it has been accessed. A recently queried segment usually has a higher temperature, whereas a segment left alone gradually cools down.
Based on my tests, temperature was not a direct predictor of query duration while the required segments remained resident. The most important part is whether the required data is resident. Once the segments were resident, their temperature could decrease significantly without changing the query duration much.
In one test, the average temperature dropped from 1.5244 to 0.0114, while all 4,374 resident segments remained resident. The same query continued to complete in approximately one second.
In this test, the observation was around 7 hours, where the temperature was close to zero; however, the later queries were still fast without the transcoding delay because most segments still remained resident.
In my currently longest observation, some column segments could still remain resident for nearly two days. Naturally, just a small chunk of the segments.
This means that a segment with almost zero temperature can still be reused immediately. Direct Lake does not need to read and transcode it again just because its temperature is low. That only becomes necessary once the segment is no longer resident.
Stage | Elapsed time | Average temperature | Resident segments | Probe duration |
Initial load | 0.0 min | 1.4260 | 3,023 | 61.66 s |
Idle checkpoint 1 | 60.2 min | 0.02196 | 3,023 | 1.90 s |
Idle checkpoint 2 | 183.0 min | 0.000332 | 3,023 | 1.80 s |
Idle checkpoint 3 | 424.4 min | 0.000000083 | 3,023 | 2.48 s |
After final probes | 425.8 min | 1.4022 | 3,023 | 1.80 s |
Tested on a semantic model with a 500-million-row fact table and several high-cardinality columns.
Eviction
Eviction happens when Direct Lake removes already loaded segments from memory. When a later query needs those segments again, they must be loaded and transcoded again, which can bring back the first-query delay.
Earlier, I mentioned three possible reasons why that could happen; let's discuss each in more detail.
The underlying data changed
When Direct Lake frames a new Delta table version, it compares the physical storage state represented by the Delta log. Existing Parquet row groups that remain part of the table can continue backing the same resident column segments. Row groups that were removed or replaced can no longer be reused, so the corresponding segments become nonresident.
The important part is therefore the physical scope of the operation.
A simple append leaves the existing Parquet files and row groups in place. Direct Lake can preserve the existing resident working set and add segments for the newly appended data.
An overwrite, update, partial rewrite, or compaction replaces existing files. Once the model frames the new Delta version, the segments backed by those removed files are no longer valid, even when the replacement files contain logically identical values.
If you would wanna be cherry-picking, this technically doesn't evict the data; you just destroy whatever was there before, so the original segments cannot be used. But I would still count it as eviction, because the result of this is that you have to load the column segments again anyway.
Storage operation | Physical change | Logical change | Residency after framing | Finding |
Append 20M rows into new files | Existing files preserved, new files added | New rows added | Existing working set preserved, total residency increased | Append did not invalidate the existing segments |
Full overwrite of the same 20M rows | All 3 files replaced by 3 new files | None | Medium workload: 147 → 21; high-cardinality workload: 105 → 21 | Replacing the files invalidated residency despite identical logical data |
Rewrite 1 of 100 buckets | 1 file removed and replaced | Identical or changed values | Numeric: 700 → 695; high cardinality: 600 → 596 | Invalidation followed the replaced file |
Rewrite 10 of 100 buckets | 10 files removed and replaced | Identical or changed values | Numeric: 700 → 650; high cardinality: 600 → 560 | A larger physical rewrite invalidated a larger scope |
Rewrite 25 of 100 buckets | 25 files removed and replaced | Identical or changed values | Numeric: 700 → 575; high cardinality: 600 → 500 | Residency loss continued to follow the rewritten scope |
Compact 250 files into 3 | 250 old files replaced by 3 new files | None | Medium workload: 1,750 → 20; high cardinality: 1,250 → 20 | Physical optimization caused almost complete invalidation |
Overwrite one dimension | Dimension files replaced | Dimension values rewritten | Dimension: 5 → 1; fact remained 500 | Invalidation remained scoped to the changed table |
The column has not been queried for a long time
When you do not query a semantic model, or you query only part of it, the untouched columns can eventually lose their resident data.
The important word here is eventually.
I repeatedly observed passive unloading across 5M, 10M, and 50M-row models on both F2 and F64. What I could not find was a fixed timeout, a useful temperature threshold, or basically any other value that would let me predict when the unloading happens consistently.
The 50M, 10M or 5M means how many fact rows the model had. Then it could have additional smaller dimensions. All models were generated and they were mix of high to low cardinality text columns and numeric columns.
Here are several tests and observations in a table; I'll try to quickly explain below it.
Test | Residency path | Total time to eviction |
50M F64 | 100% at 18.8 h → 47% at 19.3 h → 43% at 22.8 h → 0% at 46.8 h | 46.8 h |
5M F64, run 1 | 100% at 1.66 h → 94% → 61% → 27% → 7% → 0% at 2.89 h | 2.89 h |
5M F64, run 2 | 100% at 1.37 h → 84% at 1.62 h → 82% at 3.37 h → 9% → 0% at 3.89 h | 3.89 h |
5M F2, run 1 | 100% at 3.65 h → 0% at 4.15 h | 4.15 h |
5M F2, run 2 | 100% at 7.30 h → 0% at 7.55 h | 7.55 h |
10M F2, F64, run 1 | F64: 100% → 0% around 4.9 h; F2: 100% → 69% → 65% → 29% → 0% at 12.16 h | F64: 4.90 h; F2: 12.16 h |
10M F2, F64, run 2 | F2: 100% → 61% → 0% at 9.47 h; F64: 100% → 74% → 73% → 61% → 60% → 0% at 14.72 h | F2: 9.47 h; F64: 14.72 h |
5M F2 keepalive | 100% at 17.83 h → 6.9% at 18.07 h → 6.9% at 27.33 h | Untouched columns: 18.07 h; active column: not evicted by 27.33 h |
To properly explain the benchmarks and measurements, not every measurement was completely isolated on its own capacity without other operations running in the background. In the case of the F2 tests, this was usually the case; for the F64 tests, not so much. So we have to take that into account.
However, whether isolated or not, there is no clear pattern in how long the data is going to stay resident. From the tests, you could argue that bigger capacities might keep data resident for longer, but I would not claim that yet. Perhaps the more interesting finding is that keeping some columns alive can significantly postpone the total eviction.
It is also worth noting that dictionaries were usually evicted last, with several hours between the final segment eviction and the dictionary eviction. This could be meaningful because non-evicted dictionaries could technically speed up the later warm-up by around 10%, but this was only a correlation. I cannot say that it is a fact.
The only thing I can say about the tests is the eviction pattern, at least in terms of what gets evicted. In cases where there was no large drop from 100% to 0%, the first column segments to be evicted were usually high-cardinality text columns from the fact table, followed by medium- to low-cardinality columns. Numeric or dimension columns were usually the last to be evicted.
Naturally, there is no distinction in Direct Lake between fact and dimension columns from the column perspective. It is more likely the combination of high cardinality and repeated values that makes them the usual suspects to be removed first.
The engine needs to create space for more urgent columns
“Urgent columns” here does not mean any business urgency, priority, or flagging. It simply means that a query is requesting a column that is not currently resident. If there is not enough memory to frame that column while keeping everything already loaded, Direct Lake can evict some resident data segments to create space.
Each capacity SKU gives a Direct Lake semantic model a different maximum amount of data that it can page into memory. At the time of writing, the documented maximum is 3 GB for F2 and 25 GB for F64.
How much memory each column consumes is influenced by cardinality and value length, especially for text, but also by data type, value distribution, compressibility, row count, and segmentation. As a simple intuition, many repeated short values will usually require less memory than the same number of distinct long text values.
I did a looot of testing, but finding clear and always-applicable patterns was really hard. However, I did find at least one pattern that seems genuinely repeatable, but it's kinda obvious.
Recency
The cleanest pattern I found is recency. Columns that were genuinely scanned/queried shortly before the pressure had a much better chance of surviving new incoming columns. In the controlled F2 and F64 page tests, the recently scanned page retained roughly 20 to 30 more of every 100 segments than the comparison page under the same pressure.
Recency also correlated with segment temperature: the more recently used segments tended to be warmer and survive more often. I mean, this should indicate the benefit of tracking the temperature internally. However, I did not find an exact temperature threshold that would cause certain eviction. So it depends on the overall distribution of column segments and temperature.
However, this is not guaranteed; it depends on what kind of memory pressure you're dealing with, so even warmed stuff can be evicted. This is just a pattern that my tests are confident in.
Other patterns
I had several initial ideas when I began the testing. The most intuitive was that high-cardinality columns might leave sooner because they often consume more memory. In 12 historical comparisons, lower-cardinality columns survived at least as well as the medium- and high-cardinality columns.
Loading order did not produce a repeatable preference. Grouped loading looked strong in one early run, but the focused follow-up did not confirm it. Being part of an actively queried page was also unreliable when the query could be answered from cache. A genuine scan mattered more.
Either way, this is a very interesting topic for me, so I will hunt this further and report in later blog posts.
Warm DirectLake CU consumption
Edit 26.07.2026: Whole section is rewritten with now way more runs, without hitting any throttling. It also has lower candency of individual queries, I believe this explains the warm behaviour way better.
In this part, I wanted to check whether Import and Direct Lake differ in their CU consumption when running typical DAX queries, such as those generated by Power BI visualizations.
Extremely important note: these numbers cover only, let's say, best case, warm performance. There is no Import refresh cost, nor warming up for Direct Lake. This will come in its own blog post. So take this as the operational cost of fully warmed up semantic models.
For the test, I used the same data and semantic model structure across five models. On the F64 trial capacity, I tested Import, Direct Lake on OneLake, and Direct Lake on SQL. On the F2 capacity, I tested separate Direct Lake on OneLake and Import models. I assumed that Import would be the cheapest option, although I did not expect Direct Lake to differ that much. There is also not much public testing comparing the different Direct Lake variants or the same model across different capacity sizes, so these comparisons were particularly interesting.
Import provides the clearest baseline for interpreting the results. On F64, a warmed run of 48 visual-like queries consumed a median of 8.56 CU(s) in Import mode across 24 replications collected over four separate test sessions.
DirectLake on OneLake consumed 29% more than the baseline, consistently.
DirectLake on SQL is where it gets messy. It does not have one stable cost. Two-thirds of the time it lands close to OneLake, about 21% more than Import. The other third of the time it jumps to roughly 20x the baseline, with nothing in between. I checked time of day, query latency, and background activity on the capacity and Lakehouse, and none of it explains the split. It could be a silent fallback to DirectQuery, even though the capacity wasn't throttling, the pace of these queries was well within any limits, and no background jobs were interfering with the SQL portions. Either way, I ran this enough times that you can see two stable numbers, not just noise.
F2 followed a similar relative pattern to F64. Import was again the cheapest, and DirectLake on OneLake was again more expensive than Import, by about the same margin as on F64.
Note that these runs, even though using the same settings and the same kind of environment, were very inconsistent. The following table is a combination of 24 runs, each run per model around 50 queries, and the whole run spread across more than 20 hours.
The only sensible thing I could derive from this, is that the same loads can be very different, but Import being cheaper on the warmed run is consistent. Those small changes between F2 and F64 in Import mode, I wouldn't render them as dramatically different even though it could mean F2 is cheaper than F64. Still, the charging itself can occasionally spike extremely high, while the low end only dips modestly below the median, nowhere near as dramatic as the spikes upward.
Capacity | Model | Median CU(s) per window | Median CU(s) per query | Range (CU(s) per window) | vs Import |
F64 | Import | 8.56 | 0.178 | 4.30 - 217.07 | Baseline |
F64 | Direct Lake on OneLake | 11.02 | 0.230 | 7.84 - 217.17 | +29% |
F64 | Direct Lake on SQL* | 12.77 | 0.266 | 6.81 - 237.86 | +49% |
F2 | Import | 7.34 | 0.153 | 2.51 - 161.73 | Baseline |
F2 | Direct Lake on OneLake | 11.02 | 0.230 | 7.15 - 139.75 | +50% |
*I am also spreading the SQL into two layers, so you can see how it should look, but still keeping the anomalies, because it's worth noting that it can happen.
Capacity | Model | Median CU(s) per window | Median CU(s) per query | Range (CU(s) per window) | vs Import |
F64 | Direct Lake on SQL, typical(67% of runs) | 10.39 | 0.217 | 6.81 - 13.97 | +21% |
F64 | Direct Lake on SQL, elevated (33% of runs) | 181.75 | 3.786 | 165.04 - 237.86 | +2023% |
Prewarming Direct Lake
Prewarming is basically querying the semantic model before the users do, so the required column segments are already resident when they open the report. You can automate this in a Fabric notebook with Semantic Link Labs.
For example, warm_direct_lake_cache_perspective() loads the columns defined in a perspective, while warm_direct_lake_cache_isresident() restores the columns that were resident before reframing. Under the hood, these functions still send lightweight DAX queries. You can also do the same thing with sempy.fabric.evaluate_dax(), the Power BI Execute Queries REST API, or an XMLA client.
Prewarming is useful after framing, large data updates, table optimization, or before an important usage window. I would focus on the columns, measures, and relationships used by the most important report pages instead of warming the entire model, as that can waste memory and CUs. The loaded segments are not permanently pinned and can still be evicted later.
You can always validate these with INFO.STORAGETABLECOLUMNSEGMENTS() to see whether it worked as you wanted or not.
V-Order
V-Order should, in theory, improve Direct Lake performance, particularly cold column loading, because it is a write-time optimization designed to improve reads from Delta tables. However, across four of my cold-loading tests using both a large multi-table model and a smaller single-table model, V-Order consistently performed worse than the equivalent non-V-Ordered versions. I verified that V-Order was applied to the physical files, cleared the cached values before each test, confirmed zero column residency before loading, and checked full residency afterward. While this was still limited testing, I found no obvious procedural issue that would explain the results. A more thorough investigation would make an already lengthy article even longer, so I will save it for later.
For now, do not assume that V-Order will automatically improve Direct Lake performance.
CU consumption during the warm-up
I also measured how expensive a broader prewarming query can be. Before every test, I wiped out the cache, verified it, and then executed one visual-like DAX query referencing all 11 fact-table columns. I repeated the test three times per model.
Model | Average prewarming CU(s) | Median prewarming CU(s) | Previously warmed workload |
F64 Direct Lake on OneLake | 798.58 | 795.89 | 11.02 CU(s) per 48 queries |
F64 Direct Lake on SQL | 817.34 | 817.26 | 12.77 CU(s) per 48 queries |
F2 Direct Lake on OneLake | 819.94 | 819.71 | 11.02 CU(s) per 48 queries |
The semantic models were equal to the 'ordinary query' test that I did before.
The two F64 Direct Lake variants had practically identical prewarming costs. More importantly, loading all relevant fact-table columns consumed far more CU than the previously warmed workloads, even though those workloads contained 96 visual-like queries.
Comments