We first built TimescaleDB as "Postgres for time-series" for our own needs and then decided to open-source it for others. :-)
Here's some instructions on how to do so: https://github.com/slashdotdash/til/blob/master/postgres/ins...
Installing Timescaledb on Mac OS X with Postgres.app https://prathamesh.tech/2020/07/23/installing-timescaledb-on...
Not too long ago, I was asked to work on some analytics project and it required time-series data. I'm not a rockstar programmer and don't really know much about trends. So, I ended up googling and stumble upon InfluxDB. It felt like that right choice and I started playing with it. As the time passed, I realized that it might be a good software and I'm sure people love InfluxDB, but it wasn't the right choice for me. I didn't really like the docs, maybe its good now. And I had the same feeling about query syntax, it felt weird.
I moved to TimescaleDB and never looked back. I have it production for almost 2 months now. 20 tables and over 100Million writes/week. One of things I really liked was staging, I don't use docker and or anything fancy. I have bash script that and it runs on centos box and all timescale extension and postgres database are packaged together.
I was impressed by the timescale compression feature. I wasn't using it earlier because I had to be careful about what columns I need to segmentby. I would love to see some more features but I'm sure timescaledb team is already on it.
What new/other features would you like to see? (Also feel free to join slack.timescale.com or reach out at mike (at) timescale.com)
def get_escaped_measurement_name(user_name): # ... comment omitted
return '"\\"{}\\""'.format(user_name.replace('\\', '\\\\\\\\').replace('"', '\\"').replace('\n', '\\\\\\\\n'))I'll definitely be creating a ListenBrainz account. As a long time last.fm user I occasionally worry about the future of the platform. (There have been long stretches of time where it seems to have been in maintenance mode). You seem to support bulk importing last.fm data right?
We also have a Spotify importer that automatically imports stuff from Spotify, if you use Spotify, I would definitely recommend setting that up.
We're a really small team (all volunteers), so we don't move with as much urgency as I'd like to, but we've been making slow but steady progress over the years.
If you find any rough edges, or have any feedback, I'd be happy to hear, my email is in the HN profile. :)
100% opposite of what a CLI interface should be.
Timescale brings its own issues. If your goal is performance, you will be better served by clickhouse.
Plus my actual use case is even more complex, not only do I need something like MAX(temperature) > 10, I need (MAX(temperature)) > 10 && (MAX(temperature) - MAX(dewpoint)) > 4.5).
How do you delete points from a specific measurement in a specific retention policy?
For example, we have a pretty active Slack channel[0] where you can ask us anything. We've probably given away $$$$ of free support over the years ;-)
I was fully ready to just roll my own partitioned table and gave TimescaleDB a shot. It worked well. There was a bug we ran into, but it was an existing one documented on github and was addressed pretty quickly.
I still like influx, and would use it again but beware of the cardinality issues.
[1] https://medium.com/@valyala/insert-benchmarks-with-inch-infl...
postgres built-in RBAC is clunky or people would be relying on it, but an ecosystem of postgres plugins could include cleaner or smaller versions of this feature.
Even things like complex migrations (github's gh-ost, for example) could exist as DB plugins.
I'm familiar with some basics of kdb and use it often in my day job, but from what I understand that isn't widely used outside of finance?
* ClickHouse (this is a general-purpose OLAP database, but it is easy to adapt it to time series workloads)
* InfluxDB
* TimescaleDB
* M3DB
* Cortex
* VictoriaMetrics
The last three of these TSDBs support PromQL query language - the most practical query language for typical time series queries [1]. So I'd recommend starting from learning PromQL and then evaluating time series databases from the list above.
[1] https://medium.com/@valyala/promql-tutorial-for-beginners-9a...
If you are familiar with Postgres and/or SQL, then you may want to start with TimescaleDB. It's just Postgres for time-series. Full SQL, so it's possible to be productive instantly.
Object objectId objectName other...
Property propertyId objectId FK propertyName other...
Time timeId time other..
Value timeId FK propertyId FK value
Correct me if I'm wrong, but I'm fairly certain you can just write data with the same timestamp again and it gets updated. Deleting is also easily possible.
Another thing that's not mentioned in the post but was a pain point for us was that it's not easy to query for fields with "null" values [1].
I figure a lot of our pain might be because we're not as good at Influx as we are at PostgreSQL. We've been running MusicBrainz[2] for ~18 years on PostgreSQL, that knowledge will hopefully transfer over a little with Timescale.
[0]: https://github.com/influxdata/influxdb/issues/3210
[1]: https://github.com/influxdata/docs.influxdata.com/issues/717
Only works on jailbroken devices but it works well, has a local backup, and has been maintained (by me) for 2 years now.
Server costs are $2.50/mo, so this will stay up as long as I am alive.
I'd assume that most anyone building a system like this at scale has to solve these problems; does everyone roll their own?
Feel free to ask over there :)
(Btw - TimescaleDB is designed to work with Prometheus. You can see more here: https://github.com/timescale/timescale-prometheus)
Here is a page from our docs on how to perform Backup & Restore: https://docs.timescale.com/latest/using-timescaledb/backup
Not sure what's going on in that one Github issue, but we are looking into it.
The warnings are produced by COPY TO, which is used by pg_dump, since COPY TO doesn't copy chunks. It is not an issue for pg_dump, since it also do COPY TO on each chunk table.
Timescale engineer here - was part of discussion about this warning. We need to do another round and see how to remove this confusion.
That issue is now closed by the original author:
"Data is successfully dumped. also i can see the constraints, indexes are also copied successfully."
Does a WAL backup approach work?
https://docs.timescale.com/latest/using-timescaledb/backup
There are hundreds of thousands of TimescaleDB databases in production so this is generally not an issue.