LLMs are in-house trained. This is a HUGE MOAT

Delta lake is used in order to train the AI on transactional data

Basically uses Ray (I think Anyscale is the managed version) to manage the GPUs which are stored in AWS

Using real-time clickstream data is the core component that allows it to differentiate vs other systems

Think we do want data engineers for the actual data piece of AI enablement

This is important, but feels more like an "add-on" than an actual core workflow

Not super proprietary

Feel like you need actual access to historical AI data in order to determine what is most important - don't think they have the data moat to do this

This is somewhat standard transformations that a lot of data warehouses are beginning to just do themselves, and you have to compete with more enterprise governance tools like informatica, etc

This is similar to dbt - Matillion just does the compute in-warehouse but that doesn't seem like something teams would want to do

Data prep for AI itself is already being won by key players

Vectors require highly complex algorithms, etc that benefit from data flywheels. Matillion doesn't have this flywheel and it will be super hard to do it

In today's AI day and age, you need up to date real time info rather than historical batch loading. Without having a streaming broker, you aren't really playing in the game

Matillion isn't built for AI Agents to actually use APIs - it is built for data engineers to do so. Fundamentally different things

This is somewhat of a moat, but think all competitors will catch up farily quickly

Matillion is now focused on chunking, OCR, embeddings etc for vector databases

Have essentially turned into an "AI data engineer application" which is much riskier

Dominating this new AI / agentic space is key in terms of helping to "prepare" AI ready data

DBT is already starting to dominate the AI data transformation market

This takes a huge piece away from the ETL market

Dataiku's secret sauce is the LLM mesh connectors and that integration impact. Since everyone uses the same platform, within Dataiku you are able to have everyone use the same data security levels

These specific insights are what could make an enterprise-based team choose to use Dataiku because they need more specific tracking

The monitoring of SOTA model APIs is key. Think you can use open-source models with Dataiku but it is a little harder

Dataiku is more focused on the RAG pipeline, choosing data, etc as well as connecting to the tools themselves

Often times data will just be streamed directly into Estuary, but you need real-time data to aggregate somewhat histrocial events into the agentic database.

GPU will do real-time instant decisions - TigerGraph will look historically at the different lookups, etc to deliver data to the agent that it might need.

Yuga supports postgres extentions which make the ecosystem extremely robust

This is probably a big decision when actually going to market

Yuga is a little more useful - Cockroach just super stable

Yugabyte just scatters everything - makes it a little harder to query but fast writes

Cockroach is more about just being like insanely resilient - Yuga is about having more functionality while still being disributed

This is they main piece

It's not as simple as just using CDC - there are a ton of downstream issues like these schema issues that makes it harder

This delay is a big reason why having operationg and OLAP data together is useful

Leads to a flywheel affect of organizations really "living" in these type of software

This leads to a super specific data moat

This fork just points to original data for an AI agent to use in order to test features, etc

This organizing for speed is again a super optimized-piece of the data

Standard compression which is helpful

This helps speed things up a ton

This is the actual motor that looks at multiple batches of data at once. This is what allows for way more faster queries vs using a compute engine that is slower

This tiering of data enables way more efficient storage which helps teams run faster because you have less hot data to query from

This allows you to not have to send data, reducing ETL. This is massive.

Combining these is HUGE. Allows you to have dual-search and reduce hallucinations + have one source of truth.

Another huge thing of having compression algorithms that minimize data store

Able to help scale pgvector a ton compared to normal

These are real managed services that OneHouse provides. Able to optimize the OTFs in the background

Is moreso like a transformation tool for CDC

Doesn't mirror data - just converts data into text to say what it is. Is much faster than Clickhouse CDC

Is basically acting just like a translator for Clickhouse data

Could allow for faster Postgres processing, although there are bottlenecks in the data federation and it isn't as good as

Essentially Postgres is federating data from Clickhouse

Materialized views are how real-time data is able to occur so quickly

Hypercore is the multi-storage engine that allows for both OLAP and OLTP to be stored

Hypertables are the chunking / partioning of the data

Another argument is just the speed of this infrastructure in general

This is just saying which worker is working on which trace at a time so that they don't overlap on the same trace.

Metadata is what is used for organizational info, security, scores, context, user IDs, etc. The trace actually is the exact prompt sent to the AI and word-for-word response, the intermediate thoughts, tool calling, how long RAG retrieval took, etc

Braintrust combines hot data in the WAL and cold storage relatively seamlessly

This is like super similar to what Estuary is doing, but i guess custom built for super huge AI traces

This is that flywheel moat of integrating errors + convert the better answer to a golden dataset along with helping non-technical users interact with technical ones

From a use-case perspective, being able to see this level of detail is extremely helpful

Captures the behind the scenes tooling - this integrations / APIs could be a moat as well

This tool is extremely similar to what Estuary does

This is that flywheel moat of integrating errors + convert the better answer to a golden dataset along with helping non-technical users interact with technical ones

From a use-case perspective, being able to see this level of detail is extremely helpful

Captures the behind the scenes tooling - this integrations / APIs could be a moat as well

In a typical architecture, the pixel goes through Self-Attention (pixels talking to pixels) and then immediately through Cross-Attention (pixels talking to text). It’s a "chain" where:

Pixel-to-Pixel: Ensures Coherence (it looks like a real object).

Pixel-to-Text: Ensures Adherence (it looks like what you asked for).

Latent diffusion is different than using somewhat stale pieces of images. It is about compressing something down so it can later be a full sized image.

You can't just only look at historical KV caches because they won't be relevant anymore since the image has changed

The idea is that the pixels being generated by the model acts as the query. The text prompt stays the same, and the pixels ask the text which part of this should be red - then the model decides and it asks again.

The tile is loaded directly into SRAM, along with the text prompt

The attention is still all being output as vectors. The model looks at a prompt and calculates a score for how much a piece of a prompt measures for a certain tile, and then takes the actual values and multiplies them by those scores.

In each step of asking the text prompt what to do with it's pixels, every tile asks: "What do the other tiles look like, and how do we all fit the text prompt?".

These patches are what is broken down and sent to GPUs - it also needs bi-directional attention to align with both the text prompt and what other text patches are doing.

Simultaneously here is key - you don't need to determine one word to do the text. Every attention mechanism is happening at the same time.

This is the main thing - it's just open source and got a ton of adoption + network affects of people knowing how to use it, similar to Kafka

Bascially spark / flink / estuary, etc are the actual processing engines. But dbt just uses the one that is being used in the dat warehouse, so it doesn't go through a double cycle

Basically a just a language that transforms code to SQL making it easy for devs to use

This is important part of the "transformation" of tables in terms of joining them together. The keys are what matter most to keep things sane throughout the data set

It appears to be all warehouse based - use that compute. Is probably the issue in using dbt to perform in-flight transformations

This is basically done by making it so with tiling you calculate the relationship for a small tile, write down that answer and then while doing this the other tile is being pulled from the HBM so that by the time the other tile is done you already have the attention you need.

both the attention vectors and the weights needed are "tiled" in,

Together is still using NVLink - it somehow re-architects the execution and scheduling to be faster

This is the same thing Fireworks is doing

Together is getting to the low-level Kernel math similar to how Fireworks is

While the core algorithm is open, Together AI maintains a proprietary layer called the Together Kernel Collection (TKC) for their own cloud customers

This is huge - similar to how LLM models use models to predict the next word, PipeFusion uses historical parts of a tile to run another load even if the tile isn't fully finished. I imagine this would get better with more data?

Basically the tiling of the images themselves isn;t the hard part. What is hard is dertmining hoiw to make a number of patches that is not equal to the amount of GPUs. The most important part is how these tiles talk to eachother

Pipefusion's job is to decide how to chop an image into "patches" so they can be sent to different GPUs.

This inference engine isn't the multi-modal specific thing

Don't really understand this

By controlling data through the processor, you are both managing what KV cache is kept in the SRAM / which weights, as well as the actual threads on the GPU doing calculations and how much each thread should use.

CDC is often associated with real-time streaming, but batch processes use it to determine even in a batch loading process which pieces of data have changed and should be moved.

Basically just provide a guarnteed way to load in data

This schema evolution is probably pretty tough as well

Loading data isn't hard because of different file formats - oh lol

Managing these connectors is key

This is HUGE. Allows to not re-read source data which is way faster

For streaming, you have to have the architecture of maintaining the query at all times - not just writing the architecture once

The way DBT is built (warehouse centric) seems like it would take super longer compared to other stream processing frameworks

Good real-time example of needing to probably do a ton of operations to the data being streamed to do this

These machine readable formats include metadata within the data itself to make it easy for a person or BI tool to understand it

can have certain filters - processing acts as what actually sees the individual data

This is key, and similar to doing joining in a normal ELT process into a database

This is key

This unification is super important in the age of AI

This linking / joining is probably the most complex part of data transformation

Perform function son information like Excel to perform aggregate value calculations

Makes sense in terms of tools that align structure of data - kind of like excel but just at insane scale

This makes sense

Baseten isn't managing the specific things in the KV cache - just the persistence and routing of it, so it stays warm. It routes user requests to a GPU that already has specific caches. It's KV cache orchestration, not determining exactly what sits on one within one GPU.

This is a different technical approach to loading weights vs Modal - it just distributes out the entire model weights to parallelize the compute to break it down and then load those small pieces

Essentially just distributes out these GPUs when traffic requires using an Autoscaler - seems similar to Modal.

Also allows for it to be fail-proof since have multiple GPUs.

Kernel fusion itself isn't proprietary - Baseten manages open-source kernels