Видео

Fantastic! We love to hear this, and we've got lots of new content in the pipeline similar to this :)

Thanks so much! With so much interest in Apache Iceberg this year we though it was the perfect time.

We'll be having regular Q&As/AMAs in the future so there will be lots of opportunities down the line. In the meantime, feel free to reach out on the Starburst Community forum (as you have been) or on here and Lester can answer your questions in the comments.

That's amazing! We're so glad that you're finding answers the Starburst Community forum. Great to have you over at RUclips too :)

Yes, I remember it! Thanks for the comments.

Glad it was helpful!

Awesome! So glad you liked it.

Many companies actually find that an Iceberg/Icehouse architecture makes things easier to manage for the long term compared to older, alternative approaches. What kind of use case were you thinking about? We can always chat through the options with you if you're considering different angles.

Thank you! We loved the analogy too.

Glad it was helpful!

It was very interesting to see this presentation reflect efforts similar to what I undertook in my past jobs. The producers were often not tech or data savvy and lacked understanding of how downstream users leveraged the data they created - left hand not talking to the right hand. There was a significant gap between producers, the data team (aka bottlenecks), and data consumers. It became difficult to determine the ROI for data management/governance tools like Collibra as it did not directly hit bottom lines of companies. I'm excited to see this becoming a trend, as it will undoubtedly help organizations and employees easily access data, understand, trust, and use it for data-driven decision-making. I am watching this space closely. Thank you for sharing!

@@TeeA4 Thanks for sharing. We're excited too! It feels like these things are only becoming more talked about and more important.

Yeah, you can use unions instead of joins if you prefer when federating.

Thanks for reaching out. If you're using Trino and Iceberg with aws glue, I think this article might help unpack it: www.starburst.io/blog/aws-glue-iceberg-s3/ Hope it helps!

We never tire of Starburst puns around here.

Thanks! Glad you enjoyed it.

It would have been nice if the relationship between the Iceberg and S3 is explained. Because suddenly the S3 was browsed to look at the Iceberg table metadata, manifest. Thanks 👍

Glad you liked it!

Thank you so much!

Thanks!

Yes, good call out! Although Hive/Tez isn't used in data lakehouses either, Apache Tez is used in some Hive implementations of data lakes and it does reduce some of the issues associated with traditional Hive. You can think of those Hive implementations as moving a bit closer to a data lakehouse but a true lakehouse requires one of the 3 modern table formats: Iceberg, Delta Lake, or Hudi.

Awesome! Glad you enjoyed it. And we think Trino is only going to get more popular in the years to come with some of the changes happening in the data landscape.

Glad you liked it! Apache Iceberg is so big right now.

Thank you!

Glad you enjoyed it! That's a great idea for another video. Look out for something similar soon. In the meantime, you can read more about the data Icehouse here: www.starburst.io/info/icehouse/

Our Tableau extension is released for JDBC and tuned for such. You can use the Starburst ODBC Driver with Tableau and point to Starburst as generic ODBC source, but it's not generally recommended unless you can't install JDBC drivers. We have done number of optimizations and bug fixes for our Tableau extension that you won't get when using the ODBC driver, and in our experience the ODBC setup is more difficult and error-prone.

Hope that answers your question OP!

Thanks!

We love thinking about sustainable solutions when it comes to data and data architecture!

You dont need a Hive metastore. You need a metastore .. thats how Iceberg works. Trino supports Hive, Glue, Nessie, JDBC, REST and Snowflake metastores with the Iceberg connector.

This is correct! Although you can use the Hive metastore, you have many other options. This is a big part of the draw for the openness of Iceberg. You get to decide which components to use, including the metastore.

You're probably thinking that the metadata is stored on the data lake along with the data. well, it is, BUT the Iceberg spec calls out that the name of the specific metadata file that contains the current snapshot is also stored. this is crucial to the optimistic concurrency when a write happens. basically, a new version needs to be based on the prior version (or a descendant of it in certain circumstances) and the catalog aids in keeping this under control.

If you read what's on screen at that point, it says, "This is a huge oversimplification, but it's a lower bound." Explaining transistors as part of a 60s short is a little tough, but it's a good topic for a follow-up video!

Yeah, it's interesting to think about!

Glad you enjoyed it!

Thanks for commenting. This video was designed to tackle Trino's strategic displacement of similar query technologies in the ecosystem. Other videos do go into granular technical detail about how Trino works as a query engine. You might really enjoy this one for instance: ruclips.net/video/uE2rc7HxpCs/видео.html

Thanks for your comment! You're right that data products don't replace an ETL pipeline, but what they do allow is certain core ETL processes to be abstracted out from data producers/data engineering teams and packaged up as a curated "product". This product is easy to access and can be shared with data producers and data consumers alike according to access controls. This can mean different things for different organizations, but it allows some of the workload to shift to data consumer teams and frees up data engineering resources for more complex tasks. It's part of the process of "democratizing" data engineering and although it doesn't replace traditional ETL, it makes a subset of that workload more accessible to a wider number of teams. Hope that helps! Since you're interested in data products, we've got some other resources to help you explore: www.starburst.io/solutions/data-products/

Didn’t even explain what is meant by context for consumers

Glad you enjoyed the video!

Glad you enjoyed it!

So glad it was helpful for you!

Totally. Also a good description of the kinds of systems that Trino and Starburst handle best.

Glad you liked it!

So glad you enjoyed it!

Thank you. Glad you enjoyed it!

@@StarburstData I want to talk with you about your RUclips channel monetization

Perfect! Glad you enjoyed it. If you're interested in learning more about Starburst data products, we have a free course on it: academy.starburst.io/exploring-data-products

The best step-by-step walkthrough we have of Starburst and PowerBI is in our documentation: docs.starburst.io/clients/powerbi.html Hope this helps :)