Data is the new ‘oil’; however, most companies still don’t know how to mine and extract value from data, despite recognizing its potential. That’s where open source projects like Dolly come in to help users extract that value from data. In this interview recorded at the Open Source Summit (Vancouver), Denny Lee, Sr. Developer Advocate at Databricks, talks about community building and the success of the open source projects like Dolly and Delta Lake. He also talks about their partnership with the Linux Foundation, highlighting the potential benefits of being involved in a project that aligns with the success of Databricks. Lee goes on to discuss the importance of expanding the scope of the project beyond traditional contributors to include financial institutions and other companies. “There are a lot of financial companies that actually need to build foundations for securing or making their data reliable. They are talking to us about Delta Lake right now,” says Lee.
Talking about the future of the Delta Lake project, he adds that there's an ongoing debate with the community whether they should create their own foundation, Delta Lake Foundation, similar to the PyTorch Foundation or join the Linux Foundation's AI and Data organization.