Modern Data Transformation Is Powerful. Why Is Business-Ready Data Still So Hard?

xLytix Perspectives

POSITION PAPER · DATA MODELLING

Modern platforms can transform almost any data at enormous scale. The harder problem is turning technical transformation logic into reusable, governed business-ready data that more of the organisation can understand, trust and change.

10 min read · August 2026

In This Paper

We Don’t Have a Shortage of Transformation Technology
Transformation Is Not the Same as Modelling
When Business Logic Disappears Into Technical Logic
The Hidden Cost of the Modelling Layer
One Problem. Three Different Realities.
The Enterprise Challenge: Complexity
The Mid-Market Challenge: Capacity
The Smaller Organisation Challenge: Accessibility
Where xLytix Takes a Different View
One Modelling Environment. Different Ways to Work.
From Technical Tables to Business-Ready Assets
Modelling and Governance Should Not Be Separate Journeys
What Changes When AI Can Write the Transformation?
Complement the Data Platform. Don’t Replace It.
What Should Organisations Measure?
From Data Transformation to Organisational Understanding

We Don’t Have a Shortage of Transformation Technology

Modern organisations have access to extraordinarily capable data-transformation technology.

SQL remains one of the most powerful and widely understood ways of transforming data. dbt has brought software-engineering practices into SQL-based analytics engineering, where individual SQL models become reusable building blocks with dependencies between them.

Microsoft Fabric combines low-code transformation through Dataflow Gen2 with notebooks and other code-driven approaches, allowing organisations to choose between visual and technical development depending on skills and workload.

Databricks provides declarative pipelines built in SQL and Python for batch and streaming transformations, while Snowflake Dynamic Tables allow teams to describe desired transformed results using SQL and let the platform manage refreshes and dependency ordering.

The technology is not standing still either.

Semantic layers increasingly allow organisations to define important business metrics centrally rather than repeatedly recreating them inside downstream reports and applications. dbt’s Semantic Layer, for example, centralises metric definitions on top of existing models and handles joins dynamically for downstream consumers.

So the xLytix position should not be:

“Existing platforms cannot transform data.”

They can.

Extremely well.

The more important question is:

Why does turning transformed data into something the wider business can consistently understand and reuse still require so much specialist effort?

Transformation Is Not the Same as Modelling

Consider a simple transformation:

ORDERS
+
CUSTOMERS
+
PRODUCTS
+
COSTS

↓

CUSTOMER_PROFITABILITY

Technically, the transformation may be straightforward.

Join the datasets.

Filter invalid records.

Create calculated columns.

Aggregate transactions.

Materialise the result.

But the business questions are different.

What exactly constitutes a customer?

Which customer hierarchy should be used?

Is profitability calculated before or after rebates?

How are returns treated?

Which currency conversion applies?

Which date represents the transaction?

How should shared costs be allocated?

Can historical results change when master data changes?

Who owns the definition?

A transformation tells the platform how to calculate something.

A business model must also preserve what that calculation means.

Transformation creates data structures. Modelling creates reusable business understanding.

That distinction becomes increasingly important as organisations create hundreds or thousands of transformed datasets.

When Business Logic Disappears Into Technical Logic

Every transformation contains decisions.

A seemingly ordinary expression such as:

Net Revenue = Invoice Value – Discount – Returns

is no longer simply technical logic.

It is a business definition.

A join between customer and transaction data contains another decision.

A filter excluding cancelled orders contains another.

A classification turning products into product families contains another.

Over time, organisations accumulate enormous amounts of business knowledge inside:

SQL;
Python;
notebooks;
stored procedures;
ETL mappings;
BI calculations;
semantic models;
spreadsheets;
individual analysts’ knowledge.

The logic can be technically correct while remaining difficult for the wider organisation to discover or understand.

This creates a structural problem.

The business believes it has defined Customer Revenue.

The data team sees:

fct_sales_v3_final

The BI team creates:

Revenue_Adjusted

Finance maintains:

Revenue_Report_Final_v7.xlsx

All may be attempting to represent the same business concept.

The problem is no longer whether we can calculate the number.

It is whether the organisation can maintain one trusted understanding of what the number represents.

The Hidden Cost of the Modelling Layer

The visible cost of modelling is usually development.

The hidden cost is everything surrounding the model after it exists.

THE REAL COST OF DATA MODELLING

Technology

Data engineering
SQL / Python expertise
Business interpretation
Testing
Validation
Documentation
Governance
Change management
Impact analysis
Downstream reconciliation

A model rarely stays unchanged.

Business definitions evolve.

Sources change.

Columns move.

New dimensions are introduced.

Rules are corrected.

Management asks for a different hierarchy.

Regulation changes reporting requirements.

Every change creates questions:

What else uses this calculation?

Which reports are affected?

Will the model need to be rebuilt?

Who needs to approve the new definition?

Will historical results change?

A technically elegant transformation layer can therefore still create substantial organisational cost if its meaning and dependencies are difficult to understand.

A model that is easy to build but difficult to understand, reuse or change is not necessarily a low-cost model.

One Problem. Three Different Realities.

	Large Enterprise	Mid-Market	Smaller Organisation
Technology	Multiple sophisticated modelling approaches	SQL, BI, cloud and growing engineering stack	BI, SQL, spreadsheets
Data organisation	Specialist engineering and analytics teams	Small multi-skilled data team	Few/no dedicated modellers
Primary challenge	Complexity and consistency	Capacity	Accessibility
Typical symptom	Thousands of models and duplicated logic	Engineering/SQL backlog	Business logic embedded in reports
Economic issue	Maintenance and coordination	Scarce specialist time	Cost of building the capability
Opportunity	Standardise	Multiply	Enable

The modelling problem changes with the shape of the organisation.

For the Enterprise, the Problem Is Complexity

A large organisation rarely has only one modelling technology.

It may contain:

warehouse SQL;
dbt projects;
Spark pipelines;
notebooks;
stored procedures;
semantic models;
departmental data marts;
BI calculations;
legacy ETL;
regional reporting layers.

Each approach may be entirely justified.

The difficulty comes from the accumulated landscape.

A single KPI may eventually depend on transformations created across several teams.

One team owns source ingestion.

Another owns warehouse transformation.

A central analytics team maintains dimensional models.

Business units add calculations in BI.

Finance maintains additional reconciliation logic.

The organisation is not short of modelling capability.

It has too many places in which meaning can be created.

The challenge becomes:

How do we make business logic reusable, understandable and governed across a complex modelling environment?

For large enterprises, xLytix Model should therefore not be positioned simply as a visual alternative to SQL.

The more important proposition is:

Establish a more consistent modelling experience where technical transformations remain connected to their business meaning, lineage and downstream use.

For the Mid-Market, the Problem Is Capacity

Mid-sized organisations often face a different reality.

They may have a sophisticated warehouse and BI platform but only a small data team.

The same people may be responsible for:

ingesting data;
writing transformations;
creating joins;
maintaining dimensions;
implementing business calculations;
testing;
documenting;
supporting dashboards;
fixing production issues.

A request such as:

“Can we add supplier lead-time performance by product family?”

may sound straightforward.

But somebody must:

find the source;

understand the keys;

join the datasets;

define the calculation;

test it;

publish it;

update the relevant analytics;

and maintain it afterwards.

The result is an increasingly familiar bottleneck:

The organisation has the technology to answer the question, but not enough specialist capacity to answer every question quickly.

The mid-market opportunity is therefore not simply faster SQL.

It is enabling more of the modelling lifecycle to be performed through a consistent experience without sacrificing the ability to use SQL or Python when the requirement becomes more sophisticated.

The objective is to multiply the capability of the existing team rather than continually grow the specialist team around the technology.

For Smaller Organisations, the Problem Is Accessibility

Smaller organisations often do not describe what they do as “data modelling”.

But modelling is happening anyway.

It happens in:

Excel formulas.

Power Query.

BI relationships.

Calculated fields.

SQL views.

Access databases.

Individual scripts.

An analyst may combine five operational tables and create a reporting dataset.

That is modelling.

A finance manager may define margin inside a spreadsheet.

That is business logic.

A Power BI developer may build customer hierarchies inside a report.

That is modelling too.

The problem is that these models frequently become attached to the individual report or individual person who created them.

The business becomes dependent on:

the workbook

the dashboard

the person who knows how it works.

For smaller organisations, creating a formal analytics-engineering organisation may not be economically realistic.

The opportunity is therefore:

Make structured, reusable modelling accessible without requiring the organisation to first build a specialist data-engineering function.

Where xLytix Takes a Different View

xLytix starts with a different premise.

A data model should not have to choose between being easy to understand and powerful enough for engineers.

xLytix Model provides different ways of working within the same modelling lifecycle.

The current platform allows teams to create transformations visually, use SQL when more direct control is appropriate, and use Python for advanced requirements. Visual modelling supports joins, filters, calculations and transformations, while more advanced models can include unions, aggregations, derived fields, type casting, lookups and JSON flattening.

That matters because different modelling tasks have different complexity.

A business analyst creating a straightforward reporting model does not necessarily need to write code.

A SQL analyst should not be forced into a visual interface when SQL is clearer.

A data engineer should still have access to Python when the requirement genuinely needs it.

The objective is not:

no-code instead of code.

It is:

Use the level of abstraction appropriate to the problem.

One Modelling Environment. Different Ways to Work.

THE xLYTIX MODELLING EXPERIENCE

VISUAL

Join
Filter
Aggregate
Calculate
Transform

↓

SQL

Precise relational logic
Advanced transformations
Custom queries

↓

PYTHON

Advanced processing
Custom logic
Specialised transformation

↓

ONE GOVERNED MODEL LIFECYCLE

The significant point is not merely that all three approaches exist.

It is that they should create assets that participate in the same broader lifecycle.

The organisation should not have:

visual models over here

and

SQL models somewhere else

and

Python scripts nobody can connect back to the business requirement.

The model should remain understandable as part of the wider data landscape.

From Technical Tables to Business-Ready Assets

A technical modelling environment naturally asks:

What transformation do you want to perform?

A business-ready modelling environment must eventually ask broader questions:

What are we creating?

What does it represent?

At what grain does it operate?

Which entities does it describe?

Which business measures does it expose?

Which dimensions should consumers use?

Who owns it?

What depends on it?

Can it be reused?

Is it trusted?

That is the difference between creating another table and creating a business-ready data asset.

xLytix Model is designed around reusable structures including star schemas, reporting models, semantic views and other model patterns rather than treating every transformation as an isolated query.

The result should be a shift from:

raw_orders
→ sql_query_1
→ temp_customer_sales
→ final_v2
→ report_table

towards:

SOURCE DATA
      ↓
GOVERNED TRANSFORMATION
      ↓
BUSINESS MODEL
      ↓
REUSABLE MEASURES + DIMENSIONS
      ↓
ANALYTICS / PLANNING / AI / APPLICATIONS

Build once. Understand once. Reuse repeatedly.

Modelling and Governance Should Not Be Separate Journeys

One of the recurring problems in traditional data programmes is that modelling and governance happen at different times.

The model is built.

Then someone documents it.

Someone else creates lineage.

The glossary is updated later.

Ownership may be added when an audit or governance initiative demands it.

Quality rules emerge after users discover problems.

The result is predictable.

The technical asset exists before the organisation has captured enough information to understand it.

xLytix takes the opposite direction.

Its wider platform connects model transformations with lineage, quality information, business definitions, ownership and downstream use. The current xLytix product positioning explicitly connects model fields, calculations and transformations to lineage and governance context.

The important principle is:

The best time to capture the meaning of a model is while the model is being created—not months later when somebody is trying to reconstruct it.

That changes governance from a documentation exercise into part of the modelling lifecycle.

What Changes When AI Can Write the Transformation?

AI is rapidly reducing the effort required to create technical transformations.

Microsoft Fabric already uses Copilot to assist Dataflow Gen2 users with transformations through natural-language interaction, while xLytix also supports AI-assisted model creation from natural-language commands.

This creates an interesting shift.

Historically, the difficult part of a transformation may have been writing:

SELECT
    customer_id,
    SUM(revenue - discount - returns) AS net_revenue
FROM ...

AI can increasingly help create that logic.

But generating the SQL does not answer:

Should returns be calculated using transaction date or settlement date?

Which customers should be excluded?

Does Finance agree with the definition?

Can the measure be reused elsewhere?

Which downstream reports will change?

Is the underlying source trusted?

AI reduces the cost of generating technical logic.

It does not remove the need for business definition, governance and accountability.

In fact, as organisations can create transformations faster, maintaining context may become even more important.

If ten people can generate models faster than before, the organisation needs stronger—not weaker—ways of understanding what those models mean and whether they should be trusted.

Complement the Data Platform. Don’t Replace It.

As with Sync, xLytix Model should not be positioned around replacing the organisation’s underlying compute platform.

Snowflake can continue executing SQL.

Databricks can continue executing Spark.

BigQuery, Redshift, Synapse, PostgreSQL and ClickHouse can continue providing data processing and storage.

These platforms are extremely capable.

Snowflake, for example, now supports declarative SQL transformation pipelines through Dynamic Tables, while Databricks supports declarative SQL and Python pipelines.

xLytix’s opportunity sits at a different level:

Give organisations a more accessible and connected way to create, understand and govern the models executed on their chosen data infrastructure.

For some organisations, xLytix may consolidate modelling activity currently spread across several approaches.

For others, it may coexist with established engineering frameworks and provide a simpler route for particular teams and use cases.

The question should not automatically be:

“What modelling technology should we replace?”

A better question is:

“Where is our current modelling operating model creating unnecessary specialist dependency, duplicated logic or lost business context?”

The Economic Opportunity

Modelling cost is rarely dominated by the time required to type SQL.

The larger cost comes from the lifecycle around that SQL.

Understanding requirements.

Finding the right source.

Interpreting relationships.

Testing joins.

Reconciling numbers.

Documenting the model.

Explaining it to downstream teams.

Changing it later.

Investigating what broke.

Recreating similar logic elsewhere.

The economic opportunity is therefore to reduce the organisational effort surrounding the transformation.

For an enterprise:

reduce duplication and coordination cost.

For the mid-market:

increase what a constrained team can deliver.

For a smaller organisation:

make structured modelling achievable without creating a large specialist function.

That is a much more significant proposition than simply:

write transformations faster.

What Should Organisations Actually Measure?

If xLytix Model is changing the operating model, we should measure more than query execution time.

Delivery

How long does it take to turn raw source data into a reusable business-ready model?

How many models can the team deliver?

How much specialist engineering effort does a typical model require?

How long does a business change take to reach production?

Reuse

How often are common entities, calculations and dimensions reused?

How much similar business logic exists in multiple models or dashboards?

Can different teams consume the same governed model?

Quality

How much reconciliation is required after a model is published?

How frequently do users identify inconsistent calculations?

Can issues be identified before downstream consumption?

Change

Can the organisation identify downstream impact before modifying a model?

How long does it take to understand the consequences of changing a field or calculation?

How much manual regression checking is required?

Organisational Leverage

Which modelling tasks genuinely require specialist engineering?

Can analysts safely create more standard models themselves?

Are engineers spending more time solving complex problems and less time implementing repetitive business transformations?

Governance

Does the model have:

ownership?

business meaning?

lineage?

quality context?

understandable calculations?

known downstream consumers?

Measure the usefulness and maintainability of the model—not simply whether the transformation ran successfully.

From Data Transformation to Organisational Understanding

Modern data platforms have made transformation exceptionally powerful.

SQL is accessible.

Python provides enormous flexibility.

Visual tools continue to improve.

Declarative pipelines reduce operational effort.

Semantic layers are bringing consistency to important metrics.

The next challenge sits above the transformation engine.

It is making business logic understandable.

Reusable.

Governed.

Changeable.

Accessible to people with different levels of technical expertise.

And economical enough to operate across organisations of very different sizes.

That is the problem xLytix Model is designed around.

Not simply:

Can we transform the data?

But:

Can we turn technical data into reusable business understanding without making every change dependent on another specialist project?

About xLytix Model

xLytix Model helps teams transform raw data into reusable business-ready assets using the approach appropriate to the requirement.

Teams can work visually for common transformations, use SQL when direct control is preferable, and use Python for advanced requirements, while keeping models connected to the wider xLytix lifecycle of governance, analytics and operations.

Transform the data. Preserve the meaning. Reuse the understanding.

Explore xLytix Model

Continue Exploring

POSITION PAPER · DATA INTEGRATION

Data Integration Is Solved. So Why Is Getting Usable Data Still So Hard?

Modern platforms are exceptionally good at moving data. The harder problem is turning it into trusted, usable business data without excessive engineering dependency, complexity and cost.

Read Position Paper →

POSITION PAPER · DATA GOVERNANCE

Why Data Governance Must Move Into the Flow of Work

Governance becomes difficult when it is separated from the processes that create and consume data. What changes when governance becomes part of the lifecycle itself?