Dark, a New Programming Language for ‘Deployless’ Deployments
There are too many incrementalists in tech these days, according to Paul Biggar, who along with Ellen Chisa, is unveiling the Dark, a programming language aimed at making deploying software simpler.
“The complexity of building backends today is just out of control. It’s infrastructure, it’s deployment, it’s APIs… Dark is about removing all this complexity,” Biggar said.
The technology involves a holistic programming language, structured editor and infrastructure for building backend web services.
“We keep adding things, but we never take things away. AWS has 200 services and never takes any away,” he said, calling garbage collection the last really good innovation in the industry.
In a video explaining Dark’s philosophy, Biggar, who previously founded CircleCI, points to the 455 tools in the cloud native landscape map.
“To build a backend today, you need to know Kubernetes, Docker, Git, load balancers, dozens of AWS services, SQL, NoSQL, Kafka, Unix, GraphQL, gRPC, Thrift and Memcached … And you need security and optimization and networking and observability and the complexity just goes on and on and on,” he said.
He maintains you don’t need AWS or containers or Docker files or service meshes or API gateways or Kubernetes.
A Deployless Technology
The San Francisco-based company emerged from stealth at the end of July and announced $3.5 million in seed funding that it raised in 2017, but has been keeping its technology under wraps until now.
Erica Brescia, co-founder and former chief operating officer (COO) of Bitnami , now COO of GitHub (which acquired Bitnami), is among those who have invested in the company. She calls Dark “a bold vision,” adding “there are few that I believe truly have the capability to execute well on it. Ellen and Paul are a formidable team and the right founders to tackle this tremendous opportunity.”
Just as there are servers in serverless, there are deployments in deployless, you just don’t have to think about it.—Paul Biggar
Chisa explained in an interview, “When people build a developer tool, they’ll pick one specific problem to solve, then continue to work incrementally in that space. They’ll make a better version of a text editor, make a slightly different language, or make a better CI tool, make a better version control tool, but very rarely make tools that cross those boundaries.”
Dark’s language, editor and infrastructure are tightly integrated to work together to simplify things. It’s described as “deployless,” a term coined by Jessie Frazelle, a Bay Area engineer.
But just as there are servers in serverless, there are deployments in deployless, you just don’t have to think about it, Biggar said. The company touts that deploys on Dark take 50 milliseconds.
“When you write new code in a function or HTTP/event handler, we send a diff of the Abstract Syntax Tree (the representation of your code that our editor and servers use under the hood) to our servers, and then we run that code when requests come in. So deployment is just a small write to a database, which is instant and atomic. Our deploys are so fast because we trimmed a deploy to the smallest thing possible,” he explained in a blog post.
Language, Editor and Infrastructure Intersect
He calls Dark an abstraction, a language that sits over the cloud. It involves primitives familiar to those who have written backends before.
“If you’ve written anything in Rails, Node or Express or Django, these are the standard primitives,” he said.
The structured editor structures the code you write to prevent errors. It won’t let you write gibberish. It’s not possible to write syntax errors because there isn’t a parser. Every keystroke modifies the abstract syntax tree (AST), similar to Paredit, Sketch-n-Sketch, Tofu, Prune, and MPS.
The functional/imperative hybrid language is influenced by Elm, Rust, OCaml, Swift, Ruby on Rails, Clojure and TypeScript.
“You still code in a largely imperative way, but we’ve taken a lot of influences from functional languages,” he said.
“We try not to innovate too much in the language. Where we do the interesting stuff is at the boundaries – the boundary between the editor and the language, the language and the infrastructure, the language and the editor, and in particular, at the intersection of all three.”
Though they looked at existing languages, they didn’t really fit with what they were trying to do, according to Chisa.
“JavaScript is great, but it wasn’t designed for going directly to a distributed system. The way our editor works… it already knows what code you’re writing and we know that it won’t break something,” she said.
Also, if they worked with just one language then they’d be spending all of their time on adding more languages rather than building more functionality.
It uses feature flags as the key to everything. Feature flags enable developers to hide a feature, provide access to a limited set of users or immediately execute a kill switch.
Built-in Feature Flags and Monitoring
Everything is written as a feature flag, which provides safety. The feature flag creates a sandbox. All code that you’re editing is locked, with the previous version still running, and is not released until you say so. Then it can be released incrementally to a percentage of users so you can see how it works until it’s 100 percent.
Today developers can use browser-based integration tests there. In the future, Dark will add unit tests and functional tests. It will add version control.
“We want these to be a branch you can go back to in your history, just like Git,” he said.
The tight integration of language, editor and infrastructure also provides observability into your live production systems from the Dark editor. Every trace is recorded. It removes the need for separate monitoring, tracing, instrumentation, logging, and error handling.
“I love the traces feature, which allows me to see live data while creating new endpoints or making changes on the backend. There is nothing else like it out there,” said Chase Olivieri, who used Dark in creating Altitude, a subscription service providing personalized flight deals.
Birb, a project tracking tool, is another production use of Dark.
Dark is being built for existing developers who want to build backends, not newbies, Biggar wrote in answer to questions posed on Hacker News. It’s targeting two major areas: backends for client apps, such as mobile apps, or single page web apps written in React, Vue, Angular; and for building new services in existing microservices-based architectures.
It’s designed to help you build backend services with little to no scaling and maintenance overhead.
“On the other hand, if you’re building stuff like embedded systems, theorem provers, or machine learning models, Dark won’t support this sort of application for many years,” he said.
Andrius Bartulis, technical lead at Cre8ion in a tweet called Dark, “one of the most exciting and ambitious projects in the programming language/platform space right now.”
See also “Dark Deep Dive: From Editor to Infrastructure in Less Than 50 Milliseconds.”
