Hacker News Reader

Comment by killme2008 | Hacker News Reader

killme2008

Thanks for your question. GreptimeDB, like MongoDB, is schemaless. When ingesting data via OTEL or its gRPC SDKs, it automatically creates tables by inferring the schema and dynamically adds new columns as needed.

Secondly, I prefer wide tables to consolidate all sources for easy management and scalability. With GreptimeDB's columnar storage based on Parquet, unused columns don't incur storage costs.

View on HN

Thanks, that seems promising. So much of the documentation is schema-oriented, I didn't see that it supported dynamic schemas.

I find it interesting that Greptime is completely time-oriented. I don't think you can create tables without a time PK? The last time I needed log storage, I ended up picking ClickHouse, because it has no such restrictions on primary keys. We use non-time-based tables all the time, as well as dictionaries. So it seems Greptime is a lot less flexible?

Yes, GreptimeDB requires a time index column for optimized storage and querying. It's not a constraint of a primary key, but just an independent table constraint.

Could you elaborate on why you find this inconvenient? I assumed logs, for example, would naturally include a timestamp.

It's less convenient because it makes the database less general-purpose. The moment you need to go beyond time-based data, you have to reach for other tools.

ClickHouse is such a wonderful database precisely it's so incredibly flexible. While most data I interact with is time-based, I also store lots of non-time-based data there to complement the time-based tables. The rich feature set of table engines, materialized views, and dictionaries means you have a lot of different tools to pick from to design your solution. For example, to optimize ETL lookup, I use a lot of dictionaries, which are not time-based.

As an example, let's say I'm ingesting logs into Greptime and some log lines have a customer_id. I would like the final table, or least a view, to be cross-referenced with the customer so that it can include the customer's name. I suppose one would have to continually ingest customer data into a Greptime table with today's date, and then join on today's date?

Fair point. Joining time-series data with business data is often necessary. While GreptimeDB currently supports external tables for Parquet and CSV files, we plan to expand this support to include datasources like MySQL and PG in the future.

Reply on news.ycombinator.com