Data platform & lakehouse
Data platforms and lakehouses, from raw sources to Gold tables
Veraplot designs and builds data lakehouses on AWS for companies that need one reliable foundation for reporting, analytics and AI. We land your sources in Amazon S3 as Apache Iceberg tables, refine them from Bronze to Silver to Gold with dbt, Python and Spark, and govern them with DataHub and Lake Formation. Each platform is delivered by senior data engineers at a fixed price agreed up front, and you own the code and the infrastructure.
One version of the truth
Finance, operations and sales query the same Gold tables, with metrics defined once in dbt instead of re-invented in every dashboard.
Open by design
Iceberg tables on S3 are readable by Athena, Spark, Trino, Snowflake or Databricks. Changing query engine later does not mean migrating your data.
Ready for AI, not just BI
Clean, documented, versioned tables are exactly what ML models and LLM agents need. The same platform feeds dashboards, forecasts and assistants.
What you get
Target architecture and data model
Zones, domains, naming conventions and the Bronze/Silver/Gold model, documented and reviewed with your architects before the build starts.
Ingestion pipelines
Batch, CDC and streaming connectors for your priority sources, landing raw data in Bronze with full history and the ability to replay.
A dbt project with tests and docs
Silver and Gold models as versioned SQL, with automated tests, generated documentation and a semantic layer for shared business metrics.
Catalog, lineage and access control
DataHub or the Glue Data Catalog for discovery and lineage, Lake Formation for row- and column-level permissions across AWS accounts.
Infrastructure as code and runbooks
The whole platform in CDK or Terraform, with CI/CD, monitoring, alerting and runbooks your team can operate from day one.
The stack
Picked for your context, not for a partner badge. Open formats first, managed AWS services where they pay for themselves.
Catch it
Batch, CDC and streaming, straight into Bronze
- Apache Kafka
- Amazon MSK
- Amazon Kinesis
- Amazon Data Firehose
- AWS DMS
- Debezium
- Amazon AppFlow
- AWS Lambda
Keep it
Open table formats on S3, no proprietary vault
- Amazon S3
- Apache Iceberg
- Amazon S3 Tables
- Apache Parquet
Refine it
Bronze to Silver to Gold, as tested code
- dbt
- SQL
- Python
- PySpark
- Apache Spark on AWS Glue
- Amazon EMR
- Polars
- DuckDB
- pandas
Run it
Schedules, dependencies, retries and backfills
- Apache Airflow
- Amazon MWAA
- Dagster
- AWS Step Functions
- Amazon EventBridge
Trust it
Catalog, lineage, quality checks and permissions
- DataHub
- AWS Glue Data Catalog
- AWS Lake Formation
- dbt tests
- Great Expectations
- Soda
- Data contracts
- OpenLineage
Serve it
SQL, dashboards and AI-ready tables
- Amazon Athena
- Amazon Redshift
- Trino
- Power BI
- Amazon QuickSight
- Grafana
- dbt Semantic Layer
- Amazon SageMaker Feature Store
- pgvector
Typical use cases
National transport data warehouse
Public sector. A centralised warehouse and BI dashboards over national road-transport control data, with time-series analysis supporting policy-making and strategic monitoring.
Serverless data lake
Automotive. A serverless lake on Amazon S3 orchestrated with Step Functions, consolidating business sources into curated, query-ready layers for analytics teams.
Cross-account data sharing
Industrial group. Lake Formation grants expose Glue and S3 Tables (Iceberg) catalogs to Power BI and SAP consumer accounts, without copying a single file.
ERP and marketing data with a speed layer
Industrial group. SAP and Marketo ingesters land data in the lake, while a Firehose speed layer makes fresh events queryable alongside nightly batch loads.
From first call to production
- 01
Discovery call
We map your systems, constraints and the outcome that matters. You talk to the engineer who will build it, not an account manager.
- 02
Fixed-price scope
A written scope with deliverables, milestones and one price, agreed before any code is written. No day-rate meter running in the background.
- 03
Build in short iterations
Working increments every one to two weeks, demoed on your data in your AWS account. Course corrections happen early, not at the end.
- 04
Handover and optional run
Code, infrastructure as code, runbooks and a walkthrough with your team. Run it yourselves, or keep us on support month-to-month or on a 1–3 year plan.
Questions buyers ask
How much does a data lakehouse project cost?
It is a fixed price, set after a discovery call once your sources, volumes and first business domain are clear. We quote per deliverable, not per day, and our pricing typically sits 30–50% below equivalent ESN rates. A focused first domain is a medium-sized engagement; a multi-domain platform is split into milestones so you commit one stage at a time.
How long before we see data in production?
A first domain usually reaches production within one to three months: ingestion, Bronze/Silver/Gold models and a dashboard on top. A proof of concept on a single source can be live in a few weeks. Platforms covering several domains run three to six months, delivered in iterations with a demo on your own data every one to two weeks.
Do we need Databricks or Snowflake?
Not necessarily. On AWS, Iceberg tables on S3 queried with Athena, Spark on Glue or EMR, and dbt for transformations cover most analytics workloads with pay-per-use pricing. If you already run Databricks or Snowflake, we build on it; both can read Iceberg. We recommend based on your workloads, skills and budget, and we have no licence to sell you.
Data lake, data warehouse or lakehouse: which do we need?
A data lake stores raw files cheaply but is hard to trust; a warehouse is reliable but costly and closed. A lakehouse brings warehouse guarantees (ACID transactions, schema evolution, time travel) to open Iceberg tables on S3. For most companies, that means one storage layer serving BI, data science and AI, with compute you pay for only when queries run.
Why dbt, and where do Spark and Python fit?
dbt turns transformations into versioned, tested and documented SQL that analysts can read and review. Spark, through PySpark on Glue or EMR, handles what SQL should not: very large volumes, complex parsing, semi-structured files. For mid-sized data, Python with Polars or DuckDB is often faster and cheaper than either. We choose per pipeline, not per trend.