Zed is a system that makes data easier by
utilizing our new super-structured data model.
Zui ("zoo-ee") is a desktop app to explore, query,
and shape the data in your super-structured data lake.
A Tools-Based Approach
Inspired by the Unix-tools design pattern, everything Zui does
can be run from the command line. Think of Zui's components like
Lego blocks that you can easily interconnect and assemble.
It’s all open source, so have a look.
Zui is packaged as a desktop app, built with Electron just like Slack.
Once installed, you can drag your data in or use the CLI tools to load
data into a Zed lake. The Zed language, embedded in Zui, has operators
to explore the shape of your data and develop low-code "shapers" to
transform messy data into clean models.
Zui provides a rich query language that allows you to easily
perform simple, intuitive keyword searches while running rich
analytics with sophisticated filters
and pipelines. Zed is available both in the Zui editor as
well as in the
Zed CLI tools.
The ZNG Data Format
Does the world really need another data format? Yes indeed!
Zui stores all its data in the new data format
ZNG is richly typed and embeds its type schemas in the data stream.
It's inspired by Avro,
but without the hassle of out-of-band schema
definitions and schema registries.
Built on Go
The Zui backend is built from the ground up using Go, the system
language that powers massive infrastructure at the world's largest
Internet sites. In fact, when Zui runs on your desktop, there's
Zed server running in the background.
Zed and Python
Your data is everything. If you're running machine-learning experiments
or just exploring the schemas in your pipeline, Python is
your go-to environment. But when you spend all your time futzing with JSON,
waiting for it to load, or cramming messy data sets into a
relational database, you realize that you're not getting stuff done.
Zed turns this all around and makes managing data easy. With Zed,
you can spend your time doing real work instead of futzing with your
Zui was originally built as a security platform but we hit a wall
trying to proficiently support both search and analytics equally well
for eclectic security data.
After poring over all the existing tech stacks and data formats, we realized
there must be a better way and stumbled upon the concept of super-structured data.
Zui is still a great solution for security use cases, but we've been busy
generalizing the system to handle a broad range of thorny data engineering
The Zed Lake
The Zed Lake is hot off the presses. While very much a work in progress,
it's ready for you to kick the tires.
With a Zed Lake, your data can enjoy the structured-ness of relational
tables while simultaneously embracing the adaptability and schemaless-ness
of JSON. No need to define tables or schemas. Just load data into
your Zed Lake using Git-like commits.
We're not trying to make data lakes run 10X faster.
We're trying to make them 10X easier!
Whether you explore a Zed Lake with the Zui app or with the
zed command-line client, we are maniacally focused on ease of use.
If you find anything confusing or difficult, let us know and we'll
work with you to fix it!