Agenda
Agenda
Day 1
Registration Open, 2nd Floor
Coffee Break
Welcome
Joran Dirk Greef | TigerBeetle
A Systems-Minded Approach to Creating a Music Player Application
Andrew Kelley | Zig Foundation
14 years ago, I created a music player server with a web-based user interface. Since then I have used it most days of my life, but it has lately suffered from severe bitrot. Now, I am breathing life back into the project by rewriting it from scratch in Zig. A decade and a half later, I'm solving the same problems as before but with a fresh pair of eyes, a finely honed set of skills, and a new programming language. This talk compares and contrasts how the music player application was built before versus now, going into details on troubleshooting opaque audio processing bugs and how to keep the import-from-YouTube feature working despite Google's best efforts.
Distributed Asynchronous Await • A new programming model for distributed applications
Dominik Tornow | Resonate
Modern cloud-based applications are composed of countless concurrent, distributed processes pushing traditional programming models past their breaking point. Distributed Async Await, a new programming model for the cloud, extends the simplicity of async-await to tame the complexity of the cloud. Building on the foundations of functions and promises, Distributed Async Await enables “simple code that just works”. In this talk, we will explore the theoretical and practical foundations that makes this elegant approach possible. Gain a comprehensive understanding of async-await, functions, promises, and the vital concepts of concurrency and coordination in distributed systems. Take away actionable insights on how to replicate the delightful developer experience of traditional applications as you craft cloud-based, distributed applications. Design and develop cloud applications that are not only scalable and reliable but downright delightful to work on. Transform the way you approach cloud computing.
Lessons Learned from 2 Decades of Building Data Infra Products
Deepti Srivastava | Snow Leopard
Lunch (90min)
Distributed Pure Functions
Richard Feldman | Roc
There's a lot of hand-waving when it comes to scaling purely functional code. It would be easy to get the impression that as long as all your functions have no side effects, and all your data is immutable - say, because the programming language guarantees it - then distributing the work across cores and machines becomes trivial. Theoretically, a purely functional language could even parallelize workloads automatically for you behind the scenes! Naturally, things get trickier in practice. This talk dives into the practical considerations of distributing functions that are known to be pure. How much can really be done automatically? How do you avoid the accidentally slowing things down when the coordination costs get too high? What are the tradeoffs and edge cases involved? Come find out what happens when pure functions get distributed!"
Distributed DDL Replication at Global Scale
Gwen Shapira | Nile
When building SaaS, having multiple copies of the same database schema is common. Either because tenants are isolated to individual schema or for sharing. At Nile, we are building serverless Postgres for modern SaaS. We handle multi-tenant database design by isolating each tenant to a virtual Postgres database. We let users place the tenants on shared or dedicated compute nodes anywhere on the globe. The tenant virtualization design lets developers connect to each tenant database individually, query data for just this tenant, take a backup, restore, upgrade, or perform any other database operation. We aim to provide tenant isolation while still providing the admin simplicity of a single Postgres database. One of the challenges we had to address was replicating DDLs across all the tenants across the globe and across computes. When developers change their application schema, we need it to succeed or fail on every virtual tenant database and maintain a consistent state at virtually the ""same time"". We also need to preserve the transactional guarantees that Postgres extends to DDLs. In this talk we’ll discuss the approach we took at Nile, dive into key challenges and architecture decisions, and share some of the alternatives we rejected and why we chose our specific approach. You'll learn in-depth details of building Postgres extensions, Postgres locking internals and how we handle distributed transaction failures while pretending to be just a normal Postgres. "
Afternoon Break
Jepsen 15: What Even Are Transactions?
Kyle Kingsbury | Jepsen
Databases often promise "ACID transactions", but what does that phrase actually mean? Using generative testing and fault injection techniques, we'll look at transaction safety in three systems which take different approaches not only to replication, but to transaction semantics themselves. We'll see how MySQL's "Repeatable Read" is not repeatable, and how MySQL clusters hosted in AWS could routinely violate Serializability. We'll discuss RavenDB, whose "business transactions" turned out nowhere close to claimed "ACID" properties. Finally, we'll explore a system which offers serial inter-transaction but concurrent intra-transaction semantics, which challenges our intuition about operation composition under "Serializability".
Bet Against SQL: Queries as Code
James Cowling, Sujay Jayakar | Convex
Traditional wisdom is to never bet against SQL but we’re here to convince you otherwise. The app development ecosystem accretes complexity year over year as a result of decades of bad design patterns and attempts to solve problems on the wrong layers of the stack. Consistent caching, dynamic subscriptions, and end-to-end type safety can all be provided automatically when databases are built with the right abstractions: queries that are real imperative programs. We’ll give an overview of the problems facing modern application developers and how these can be solved by a better database. We’ll also dive into some of the details and challenges when implementing this stack from the ground up at Convex.
Wrap up
Jihyun Tornow | TigerBeetle
Opening Night Event at Roof at Park South*
125 E 27th St, New York, NY 10016 *Exclusively for Speakers and Premium Ticket Holders
Day 1
Registration Open, 2nd Floor
Coffee Break
A Systems-Minded Approach to Creating a Music Player Application
Andrew Kelley | Zig Foundation
14 years ago, I created a music player server with a web-based user interface. Since then I have used it most days of my life, but it has lately suffered from severe bitrot. Now, I am breathing life back into the project by rewriting it from scratch in Zig. A decade and a half later, I'm solving the same problems as before but with a fresh pair of eyes, a finely honed set of skills, and a new programming language. This talk compares and contrasts how the music player application was built before versus now, going into details on troubleshooting opaque audio processing bugs and how to keep the import-from-YouTube feature working despite Google's best efforts.
Distributed Asynchronous Await • A new programming model for distributed applications
Dominik Tornow | Resonate
Modern cloud-based applications are composed of countless concurrent, distributed processes pushing traditional programming models past their breaking point. Distributed Async Await, a new programming model for the cloud, extends the simplicity of async-await to tame the complexity of the cloud. Building on the foundations of functions and promises, Distributed Async Await enables “simple code that just works”. In this talk, we will explore the theoretical and practical foundations that makes this elegant approach possible. Gain a comprehensive understanding of async-await, functions, promises, and the vital concepts of concurrency and coordination in distributed systems. Take away actionable insights on how to replicate the delightful developer experience of traditional applications as you craft cloud-based, distributed applications. Design and develop cloud applications that are not only scalable and reliable but downright delightful to work on. Transform the way you approach cloud computing.
Lunch (90min)
Distributed Pure Functions
Richard Feldman | Roc
There's a lot of hand-waving when it comes to scaling purely functional code. It would be easy to get the impression that as long as all your functions have no side effects, and all your data is immutable - say, because the programming language guarantees it - then distributing the work across cores and machines becomes trivial. Theoretically, a purely functional language could even parallelize workloads automatically for you behind the scenes! Naturally, things get trickier in practice. This talk dives into the practical considerations of distributing functions that are known to be pure. How much can really be done automatically? How do you avoid the accidentally slowing things down when the coordination costs get too high? What are the tradeoffs and edge cases involved? Come find out what happens when pure functions get distributed!"
Distributed DDL Replication at Global Scale
Gwen Shapira | Nile
When building SaaS, having multiple copies of the same database schema is common. Either because tenants are isolated to individual schema or for sharing. At Nile, we are building serverless Postgres for modern SaaS. We handle multi-tenant database design by isolating each tenant to a virtual Postgres database. We let users place the tenants on shared or dedicated compute nodes anywhere on the globe. The tenant virtualization design lets developers connect to each tenant database individually, query data for just this tenant, take a backup, restore, upgrade, or perform any other database operation. We aim to provide tenant isolation while still providing the admin simplicity of a single Postgres database. One of the challenges we had to address was replicating DDLs across all the tenants across the globe and across computes. When developers change their application schema, we need it to succeed or fail on every virtual tenant database and maintain a consistent state at virtually the ""same time"". We also need to preserve the transactional guarantees that Postgres extends to DDLs. In this talk we’ll discuss the approach we took at Nile, dive into key challenges and architecture decisions, and share some of the alternatives we rejected and why we chose our specific approach. You'll learn in-depth details of building Postgres extensions, Postgres locking internals and how we handle distributed transaction failures while pretending to be just a normal Postgres. "
Afternoon Break
Jepsen 15: What Even Are Transactions?
Kyle Kingsbury | Jepsen
Databases often promise "ACID transactions", but what does that phrase actually mean? Using generative testing and fault injection techniques, we'll look at transaction safety in three systems which take different approaches not only to replication, but to transaction semantics themselves. We'll see how MySQL's "Repeatable Read" is not repeatable, and how MySQL clusters hosted in AWS could routinely violate Serializability. We'll discuss RavenDB, whose "business transactions" turned out nowhere close to claimed "ACID" properties. Finally, we'll explore a system which offers serial inter-transaction but concurrent intra-transaction semantics, which challenges our intuition about operation composition under "Serializability".
Bet Against SQL: Queries as Code
James Cowling, Sujay Jayakar | Convex
Traditional wisdom is to never bet against SQL but we’re here to convince you otherwise. The app development ecosystem accretes complexity year over year as a result of decades of bad design patterns and attempts to solve problems on the wrong layers of the stack. Consistent caching, dynamic subscriptions, and end-to-end type safety can all be provided automatically when databases are built with the right abstractions: queries that are real imperative programs. We’ll give an overview of the problems facing modern application developers and how these can be solved by a better database. We’ll also dive into some of the details and challenges when implementing this stack from the ground up at Convex.
Wrap up
Jihyun Tornow | TigerBeetle
Opening Night Event at Roof at Park South*
125 E 27th St, New York, NY 10016 *Exclusively for Speakers and Premium Ticket Holders
Day 1
Registration Open, 2nd Floor
Coffee Break
Welcome
Joran Dirk Greef | TigerBeetle
A Systems-Minded Approach to Creating a Music Player Application
Andrew Kelley | Zig Foundation
14 years ago, I created a music player server with a web-based user interface. Since then I have used it most days of my life, but it has lately suffered from severe bitrot. Now, I am breathing life back into the project by rewriting it from scratch in Zig. A decade and a half later, I'm solving the same problems as before but with a fresh pair of eyes, a finely honed set of skills, and a new programming language. This talk compares and contrasts how the music player application was built before versus now, going into details on troubleshooting opaque audio processing bugs and how to keep the import-from-YouTube feature working despite Google's best efforts.
Distributed Asynchronous Await • A new programming model for distributed applications
Dominik Tornow | Resonate
Modern cloud-based applications are composed of countless concurrent, distributed processes pushing traditional programming models past their breaking point. Distributed Async Await, a new programming model for the cloud, extends the simplicity of async-await to tame the complexity of the cloud. Building on the foundations of functions and promises, Distributed Async Await enables “simple code that just works”. In this talk, we will explore the theoretical and practical foundations that makes this elegant approach possible. Gain a comprehensive understanding of async-await, functions, promises, and the vital concepts of concurrency and coordination in distributed systems. Take away actionable insights on how to replicate the delightful developer experience of traditional applications as you craft cloud-based, distributed applications. Design and develop cloud applications that are not only scalable and reliable but downright delightful to work on. Transform the way you approach cloud computing.
Lessons Learned from 2 Decades of Building Data Infra Products
Deepti Srivastava | Snow Leopard
Lunch (90min)
Distributed Pure Functions
Richard Feldman | Roc
There's a lot of hand-waving when it comes to scaling purely functional code. It would be easy to get the impression that as long as all your functions have no side effects, and all your data is immutable - say, because the programming language guarantees it - then distributing the work across cores and machines becomes trivial. Theoretically, a purely functional language could even parallelize workloads automatically for you behind the scenes! Naturally, things get trickier in practice. This talk dives into the practical considerations of distributing functions that are known to be pure. How much can really be done automatically? How do you avoid the accidentally slowing things down when the coordination costs get too high? What are the tradeoffs and edge cases involved? Come find out what happens when pure functions get distributed!"
Distributed DDL Replication at Global Scale
Gwen Shapira | Nile
When building SaaS, having multiple copies of the same database schema is common. Either because tenants are isolated to individual schema or for sharing. At Nile, we are building serverless Postgres for modern SaaS. We handle multi-tenant database design by isolating each tenant to a virtual Postgres database. We let users place the tenants on shared or dedicated compute nodes anywhere on the globe. The tenant virtualization design lets developers connect to each tenant database individually, query data for just this tenant, take a backup, restore, upgrade, or perform any other database operation. We aim to provide tenant isolation while still providing the admin simplicity of a single Postgres database. One of the challenges we had to address was replicating DDLs across all the tenants across the globe and across computes. When developers change their application schema, we need it to succeed or fail on every virtual tenant database and maintain a consistent state at virtually the ""same time"". We also need to preserve the transactional guarantees that Postgres extends to DDLs. In this talk we’ll discuss the approach we took at Nile, dive into key challenges and architecture decisions, and share some of the alternatives we rejected and why we chose our specific approach. You'll learn in-depth details of building Postgres extensions, Postgres locking internals and how we handle distributed transaction failures while pretending to be just a normal Postgres. "
Afternoon Break
Jepsen 15: What Even Are Transactions?
Kyle Kingsbury | Jepsen
Databases often promise "ACID transactions", but what does that phrase actually mean? Using generative testing and fault injection techniques, we'll look at transaction safety in three systems which take different approaches not only to replication, but to transaction semantics themselves. We'll see how MySQL's "Repeatable Read" is not repeatable, and how MySQL clusters hosted in AWS could routinely violate Serializability. We'll discuss RavenDB, whose "business transactions" turned out nowhere close to claimed "ACID" properties. Finally, we'll explore a system which offers serial inter-transaction but concurrent intra-transaction semantics, which challenges our intuition about operation composition under "Serializability".
Bet Against SQL: Queries as Code
James Cowling, Sujay Jayakar | Convex
Traditional wisdom is to never bet against SQL but we’re here to convince you otherwise. The app development ecosystem accretes complexity year over year as a result of decades of bad design patterns and attempts to solve problems on the wrong layers of the stack. Consistent caching, dynamic subscriptions, and end-to-end type safety can all be provided automatically when databases are built with the right abstractions: queries that are real imperative programs. We’ll give an overview of the problems facing modern application developers and how these can be solved by a better database. We’ll also dive into some of the details and challenges when implementing this stack from the ground up at Convex.
Wrap up
Jihyun Tornow | TigerBeetle
Opening Night Event at Roof at Park South*
125 E 27th St, New York, NY 10016 *Exclusively for Speakers and Premium Ticket Holders
Day 2
Day 2
Registration Open, 2nd Floor
Coffee Break
Registration Open, 2nd Floor
Coffee Break
Registration Open, 2nd Floor
Coffee Break
Welcome
Jihyun Tornow | TigerBeetle
Welcome
Jihyun Tornow | TigerBeetle
Welcome
Jihyun Tornow | TigerBeetle
Systems Resurgent?
Amod Malviya | Udaan
The past 2 decades has seen technology take the center stage of life in modern society. Progress in accessibility of software development and open source, allowed one to produce useful software without necessarily understanding how systems worked underneath. Using anecdotes from my professional life, I do a review of the effect of these past developments on engineering culture, and make an argument for why systems engineering is on a path of resurgence, emerging both stronger and more accessible, and how better abstractions on the systems side will lead to better engineering culture downstream, leading to better product design and experience for the end customers.
Systems Resurgent?
Amod Malviya | Udaan
The past 2 decades has seen technology take the center stage of life in modern society. Progress in accessibility of software development and open source, allowed one to produce useful software without necessarily understanding how systems worked underneath. Using anecdotes from my professional life, I do a review of the effect of these past developments on engineering culture, and make an argument for why systems engineering is on a path of resurgence, emerging both stronger and more accessible, and how better abstractions on the systems side will lead to better engineering culture downstream, leading to better product design and experience for the end customers.
Systems Resurgent?
Amod Malviya | Udaan
The past 2 decades has seen technology take the center stage of life in modern society. Progress in accessibility of software development and open source, allowed one to produce useful software without necessarily understanding how systems worked underneath. Using anecdotes from my professional life, I do a review of the effect of these past developments on engineering culture, and make an argument for why systems engineering is on a path of resurgence, emerging both stronger and more accessible, and how better abstractions on the systems side will lead to better engineering culture downstream, leading to better product design and experience for the end customers.
Being Our Own Worst Customer
Brian Lagoda, John Murray | Antithesis
How do you deterministically test a deterministic testing system? After years of handpicking customers that were a good fit for our service, Antithesis faced our toughest customer yet: ourselves. We'll talk about why we did it, what went wrong, and what we learned along the way.
Being Our Own Worst Customer
Brian Lagoda, John Murray | Antithesis
How do you deterministically test a deterministic testing system? After years of handpicking customers that were a good fit for our service, Antithesis faced our toughest customer yet: ourselves. We'll talk about why we did it, what went wrong, and what we learned along the way.
Being Our Own Worst Customer
Brian Lagoda, John Murray | Antithesis
How do you deterministically test a deterministic testing system? After years of handpicking customers that were a good fit for our service, Antithesis faced our toughest customer yet: ourselves. We'll talk about why we did it, what went wrong, and what we learned along the way.
5 Lessons from 5 Years of Building Databases at Scale
Sammy Steele | Figma
What is the fastest way to divide a group of systems engineers? Ask them to choose the best database. What actually unites most systems engineers? The idea that databases are hard. The CAP theorem isn’t just a theory. Database engineers deal with the realities of these trade offs every day. When debugging outages, they often simultaneously encounter scaling bottlenecks, availability impacts, network outages and data corruption. When a major website goes down, there’s a good chance that databases are implicated. Big risks come with big impact. You can remove existential risks to your company’s business by scaling out your databases. You can ensure customers keep buying your product by improving database reliability and consistency. Databases can be both a major source of cost savings and a major driver of new business new revenue. Come explore lessons learned from 5 years of building databases at petabyte scale at Dropbox and Figma.
5 Lessons from 5 Years of Building Databases at Scale
Sammy Steele | Figma
What is the fastest way to divide a group of systems engineers? Ask them to choose the best database. What actually unites most systems engineers? The idea that databases are hard. The CAP theorem isn’t just a theory. Database engineers deal with the realities of these trade offs every day. When debugging outages, they often simultaneously encounter scaling bottlenecks, availability impacts, network outages and data corruption. When a major website goes down, there’s a good chance that databases are implicated. Big risks come with big impact. You can remove existential risks to your company’s business by scaling out your databases. You can ensure customers keep buying your product by improving database reliability and consistency. Databases can be both a major source of cost savings and a major driver of new business new revenue. Come explore lessons learned from 5 years of building databases at petabyte scale at Dropbox and Figma.
5 Lessons from 5 Years of Building Databases at Scale
Sammy Steele | Figma
What is the fastest way to divide a group of systems engineers? Ask them to choose the best database. What actually unites most systems engineers? The idea that databases are hard. The CAP theorem isn’t just a theory. Database engineers deal with the realities of these trade offs every day. When debugging outages, they often simultaneously encounter scaling bottlenecks, availability impacts, network outages and data corruption. When a major website goes down, there’s a good chance that databases are implicated. Big risks come with big impact. You can remove existential risks to your company’s business by scaling out your databases. You can ensure customers keep buying your product by improving database reliability and consistency. Databases can be both a major source of cost savings and a major driver of new business new revenue. Come explore lessons learned from 5 years of building databases at petabyte scale at Dropbox and Figma.
Lunch (90min)
Lunch (90min)
Lunch (90min)
Lightning Talks
Natalie Vais | Spark Capital
Lightning Talks
Natalie Vais | Spark Capital
Garden of Forking Paths
Alex Petrov | Apple
Traditional randomized testing, both in the field of Databases and outside of it, might be a somewhat inefficient way to hunt for bugs. However, in this talk, we’re going to empower you with the knowledge of how to make your tests cover more grounds. By controlling the inputs of random number generation, we can make our software try less of the same thing and instead, explore new, creative ways to misbehave. This approach can substantially enhance your testing efficiency. In Apache Cassandra, we have a treasure trove of tools that beautifully stack on each other, providing unique ways to emulate real-world cluster behavior in a single JVM. The bottom layer is an in-JVM distributed test framework, with a deterministic simulator driving the system under test and Harry, a generator, and a test oracle with minimal state requirements. We’ll delve into how we've been using this tooling to unearth issues in Apache Cassandra, what we've discovered and learned, and how other database systems can reap the benefits of this knowledge. Get ready to be excited about the possibilities it opens up for your testing.
Garden of Forking Paths
Alex Petrov | Apple
Traditional randomized testing, both in the field of Databases and outside of it, might be a somewhat inefficient way to hunt for bugs. However, in this talk, we’re going to empower you with the knowledge of how to make your tests cover more grounds. By controlling the inputs of random number generation, we can make our software try less of the same thing and instead, explore new, creative ways to misbehave. This approach can substantially enhance your testing efficiency. In Apache Cassandra, we have a treasure trove of tools that beautifully stack on each other, providing unique ways to emulate real-world cluster behavior in a single JVM. The bottom layer is an in-JVM distributed test framework, with a deterministic simulator driving the system under test and Harry, a generator, and a test oracle with minimal state requirements. We’ll delve into how we've been using this tooling to unearth issues in Apache Cassandra, what we've discovered and learned, and how other database systems can reap the benefits of this knowledge. Get ready to be excited about the possibilities it opens up for your testing.
Garden of Forking Paths
Alex Petrov | Apple
Traditional randomized testing, both in the field of Databases and outside of it, might be a somewhat inefficient way to hunt for bugs. However, in this talk, we’re going to empower you with the knowledge of how to make your tests cover more grounds. By controlling the inputs of random number generation, we can make our software try less of the same thing and instead, explore new, creative ways to misbehave. This approach can substantially enhance your testing efficiency. In Apache Cassandra, we have a treasure trove of tools that beautifully stack on each other, providing unique ways to emulate real-world cluster behavior in a single JVM. The bottom layer is an in-JVM distributed test framework, with a deterministic simulator driving the system under test and Harry, a generator, and a test oracle with minimal state requirements. We’ll delve into how we've been using this tooling to unearth issues in Apache Cassandra, what we've discovered and learned, and how other database systems can reap the benefits of this knowledge. Get ready to be excited about the possibilities it opens up for your testing.
Afternoon Break
Afternoon Break
Afternoon Break
Rethinking Authentication
Frank Denis | Fastly
In distributed systems, authentication is vital for ensuring security. However, traditional systems frequently struggle to provide adequate performance, and they often remain vulnerable to real-world attacks. We’re going to explore simple and practical authentication schemes that address these limitations, significantly enhancing performance, latency, and privacy while strengthening overall security.
Rethinking Authentication
Frank Denis | Fastly
In distributed systems, authentication is vital for ensuring security. However, traditional systems frequently struggle to provide adequate performance, and they often remain vulnerable to real-world attacks. We’re going to explore simple and practical authentication schemes that address these limitations, significantly enhancing performance, latency, and privacy while strengthening overall security.
Rethinking Authentication
Frank Denis | Fastly
In distributed systems, authentication is vital for ensuring security. However, traditional systems frequently struggle to provide adequate performance, and they often remain vulnerable to real-world attacks. We’re going to explore simple and practical authentication schemes that address these limitations, significantly enhancing performance, latency, and privacy while strengthening overall security.
Durability and the Art of Consensus
Joran Dirk Greef | TigerBeetle
Which letter in ACID, if you pull it out, would undo the others? And if you put it back, would set you on the path, not only of replication, but (surprisingly) consensus too? Let's push past the limits, over-specifications and assumptions of popular protocols, to build a backup system from first principles. Recommended pre-reads: https://www.usenix.org/system/files/conference/fast18/fast18-alagappan.pdf https://pmg.csail.mit.edu/papers/vr-revisited.pdf
Durability and the Art of Consensus
Joran Dirk Greef | TigerBeetle
Which letter in ACID, if you pull it out, would undo the others? And if you put it back, would set you on the path, not only of replication, but (surprisingly) consensus too? Let's push past the limits, over-specifications and assumptions of popular protocols, to build a backup system from first principles. Recommended pre-reads: https://www.usenix.org/system/files/conference/fast18/fast18-alagappan.pdf https://pmg.csail.mit.edu/papers/vr-revisited.pdf
Durability and the Art of Consensus
Joran Dirk Greef | TigerBeetle
Which letter in ACID, if you pull it out, would undo the others? And if you put it back, would set you on the path, not only of replication, but (surprisingly) consensus too? Let's push past the limits, over-specifications and assumptions of popular protocols, to build a backup system from first principles. Recommended pre-reads: https://www.usenix.org/system/files/conference/fast18/fast18-alagappan.pdf https://pmg.csail.mit.edu/papers/vr-revisited.pdf
Wrap up
Joran Dirk Greef | TigerBeetle
Wrap up
Joran Dirk Greef | TigerBeetle
Follow @TigerBeetleDB for updates! Questions? Contact Jihyun Tornow