A Tale of Two Approaches to Decentralized Data Integrity

Holochain is quite different from blockchain, but because they are designed to solve some of the same problems — and because people try to understand Holochain in terms of blockchain all the time — we figured it would be a good idea to frame at least one key aspect of Holochain in comparison to blockchain.

A complete primer on Holochain and blockchain would need a good deal of detail about what blockchain really is and how it works, and we’d probably be addressing a lot of common technical misconceptions about blockchain in the process. This is not that article.

Instead, this piece focuses on the approach each technology takes to solving an important challenge, which is really the fundamental challenge of decentralized computing: how to ensure that data is accurate and tamper-proof in a way that is efficient enough to scale.

We’ll look at blockchain’s approach, then Holochain’s.

Blockchain and Global Consensus

Blockchain is a cryptographically secured, decentralized ledger of data. You can think of a blockchain as a record of events: the things people said, who agreed to what, who sent money to whom, and so on. Up until blockchain was invented, these sorts of records were pretty much always stored in centralized databases, such as those held by government entities or private companies. Blockchain was created as a way for people to interact and transact without needing to trust such intermediaries.

What makes blockchain secure and trustable — in other words, what ensures its data integrity — is that the data is not just cryptographically protected but also replicated to many different computers (called nodes) controlled by different people or organizations. Only when a piece of data is adopted by the multitude of nodes is it considered factual, at which point it’s committed to the record. For someone to alter the record, they would have to not only break cryptographic barriers but also change most of the copies that are floating out there — a pretty much impossible task.

The way that nodes reach ‘consensus’ on what data to commit to the record varies from blockchain to blockchain, but it typically involves some type of competition among the nodes to write the next chunk of entries, or ‘block’, to the chain. Ultimately the selection of the winning node is random, so it’s not exactly consensus in the way that people mean the term in the real world. But the important takeaway is that, one way or another, the blockchain nodes come to terms on a global state of data, where all nodes hold a replica of all the same data.

And so here we come to the scalability problem: it requires tremendous computing work for all the nodes to write and hold the same data. This makes blockchains notoriously slow processors: the Bitcoin network processes just a few transactions per second, while the Ethereum network currently processes dozens. Users are accustomed to waiting minutes, sometimes up to an hour, for a single transaction to be confirmed.

The problem gets worse as you try to make blockchain do more things, which has been its aspiration, more or less, since Ethereum positioned itself as a decentralized world computer when it launched in 2014. That’s the point at which blockchains began to be able to store not just transaction records but all kinds of files and even executable application code that performs functions when accessed, resulting in new data that also gets written to the blockchain. It’s been an attractive idea to imagine that much of what we do on the web today could be hosted on blockchain networks as decentralized applications (dApps). In reality, though, apps like social networks, communication platforms, travel-booking systems, ride-sharing systems, calendar systems, and so on need much faster throughput, by many orders of magnitude, than blockchain can provide. Can you imagine waiting several minutes (and paying a gas fee!) to get a message through on a chat platform? Or for an edit to show up on a collaborative editing tool like Google Docs? Can you imagine how much computation and storage would be required to accommodate on a blockchain all of the photos and videos on social media, with all nodes needing to write all of that data and keep it forever? It doesn’t work. Facebook receives over 4 million likes every minute and currently stores over 250 billion photos.

Efficiency is such a challenge in blockchain that there are entire companies, including some of the most talked-about crypto projects today, dedicated to figuring out how to make blockchain scale better. Some of them focus on ‘layer-1’ solutions, which attempt to increase the throughput of blockchain protocols themselves, while others are ‘layer-2’, which perform computations or store data off-chain and then periodically merge records into the blockchain. Most of these solutions, though, seem to be setting their sights on low-throughput applications such as financial transacting as opposed to live collaboration apps, social networks, media platforms, and so on. And the few that do seem capable of handling a broader set of applications make compromises on decentralization, concentrating hosting and consensus mechanisms among centrally authorized nodes.

Still, the accomplishment of blockchain is not to be taken lightly. It’s more resilient to corruption than any ledgering or value-storage system than has ever existed before, and it is changing global finance as a result, with lots of room for growth still. But what’s probably even more important is the awareness it has sparked of what’s possible. Its aspirations have infected broad communities of people with a sense that we could communicate and transact without centralized intermediaries. Blockchain’s scalability challenges may ultimately limit its utility, but it has already revolutionized how humans think about interacting.

Holochain and Embodied Local States

The architects of Holochain began with a basic question: what if everyone could actually just hold their own data and share it with the network as needed? If everyone could just host themselves rather than relying on mining nodes to do it? We could avoid all this massive replication, which would obviously be much more efficient. We would just need to do it in a way that still ensures data integrity. We would have to be completely confident that, as everyone represents their own data to the network, there is no way for people to mis represent their data.

That is fundamentally what Holochain does. Holochain is a framework for ensuring data integrity within a decentralized application without relying on anyone other than the users themselves.

A Natural Solution

At this point in the conversation, people familiar with blockchain are often skeptical. What’s to prevent people from lying about their state? From, say, spending the same money in two different places? (Holochain supports applications far more diverse than just currencies, but it’s often useful to use currencies as an illustration.)

We’ll get to some of the mechanics that make this possible in a moment. First, let’s look at the principles behind the mechanics, by way of analogy to nature… starting with some of its smallest objects.

Consider the covalent bonding of a chlorine atom and a hydrogen atom to create a molecule of hydrogen chloride. This requires the hydrogen atom to have a free electron available, i.e. not shared with any other atom. How does the chlorine atom ‘know’ whether the hydrogen atom has an electron available? It’s simply apparent. The hydrogen atom embodies whether a free electron exists in its state. It’s not able to misrepresent whether there’s a free electron, and it’s not able to ‘double-spend’ its electron, because the availability of an electron is evident to other relevant atoms upon inspection. There is global visibility, on demand, of local state.

It would be ridiculous to believe that, in order to know whether an atom has a free electron, there should be a global, synchronized ledger of the whereabouts of all electrons in the universe. Or — to use a natural example with somewhat larger objects — that the status of the trillions of cells in our bodies should be registered on a global body tracking system. The cells already embody the changes: the levels of oxygen in the blood cells, for example, determine whether they offer oxygen to organ tissue cells in exchange for carbon dioxide. Then the reverse happens once the blood cells reach the lungs, where they exchange carbon dioxide for fresh oxygen. These interactions are determined on a cell-to-cell basis, without reference to any body-wide ledger of blood cell oxygen levels.

Holochain’s premise is that it’s equally unnecessary for all nodes in a decentralized application to hold a record of everyone’s state, as happens in blockchain, or for nodes to reach consensus before a user commits a state change to their own record. The local embodiment of state can act as its own authority, as long as the structure of data is tamper-proof. Also, only information necessary for larger-scale coordination needs to be widely shared, with all shared data strongly tied to where it came from. In this way, Holochain is an agent-centric system for decentralized computing: the users (agents) themselves are the definitive source of information in the system.

Okay, with those principles established, let’s look briefly at some of the architecture that makes Holochain’s data structure tamper-proof and scalable. After a cartoon break, that is.

Holochain’s Core Components

Source Chain. Each user hosts their own data on a source chain, which is a cryptographically signed record of everything you’ve ever done or said within Holochain applications, stored locally on your machine. Source chains, like blockchains, are hash chains, which associate a cryptographic fingerprint (or ‘hash’) with every record (or ‘entry’). Hashes are unique to the particular data they represent: changing just one comma to a period in a thousand-page book would result in a completely different hash.

DHT. Data that needs to be shared with the network is published to a shared environment called a distributed hash table, or DHT. Your tweets and comments in a Twitter-like app, your ride requests in an Uber-like app, your edits in a collaborative document editor… all of these are on the DHT. (Data that doesn’t need to be shared can remain private to your source chain.) Each user running a Holochain app stores a tiny slice of the app’s DHT, in addition to hosting their own data.

DNA. Each application’s rules for sharing data are written into the application code itself, known as DNA. The DNA is what says that this is an application for tweeting (which involves sharing data with a certain structure) versus calling rides or co-editing documents (which involve sharing different data with different structures). It also defines who can join the app’s network: can anyone join, do you need an invitation code or to pay, or is there some other criteria?A copy of the DNA is hosted by every application user, which means that any user is able to validate whether data being shared to their slice of the DHT conforms to the application’s rules.

But What Makes It Tamper-Proof?

Okay, cool structure maybe, but why can’t someone simply alter their source chain and misrepresent their data to others?

You can think of a source chain like a diary: each page contains a header, which identifies the fact that something happened and when it happened, and an entry, which contains the content of what happened (such as “I sent 100 units of currency to so-and-so”). Some of these entries might have been published to the DHT and others might not have, but in all cases the headers, which contain the hashes of the entries, are shared to the DHT. In other words, I may or may not have published the contents of a given diary page, but everyone is able to see that I wrote something on the page, and they are able to see the unique fingerprint that corresponds the contents of the page (which would completely change if I were to ever modify the contents even slightly).

Let’s say you and I are doing some transaction such that I need to send you 250 units. The app’s rules (encoded in the DNA) will say that in order for this transaction to go through, you need to verify my account balance, which means that I need to show you enough information from my diary in order for you to do so. (Remember, there is global visibility of local state, to whatever degree is necessary for a given action to be validated.) Your computer can very quickly add up all the pluses and minuses on the pages in my transaction diary, my source chain. You know that I’m not hiding any pages because you can check the DHT and see exactly how many pages have writing on them. And there’s no way I could have altered a previous page without making it obvious I’ve done so, because every action I take is a new timestamped event with a new header and new unique hash that also gets shared to the DHT. Plus a system of header monitoring by ‘neighbors’ ensures that I’m unable to fork or roll back my source chain without getting flagged. If anything doesn’t add up, or if it seems like something has been obscured, the transaction simply fails the validation rules and does not take place.

But What Makes It Scalable?

Most of blockchain’s challenges with scalability are really challenges of managing global consensus. Since Holochain maintains data integrity without the need for consensus, it doesn’t run into the same limitations.

There is no need for universal agreement. Keeping with our currency example for a moment: how many computers need to confirm our transaction in order for it to be executed? If this were blockchain, all nodes would need to come to terms with one another and keep a record of our transaction forever. In Holochain, the transaction is complete when just two computers have written it: yours and mine. Then, afterward, we publish the data to the DHT, and randomized groups of nodes store it so that others can confirm for themselves, later as the need arises, that we’re representing our states accurately. Data validation is party-to-party, just like for all the cells in our body, just like for all the atoms in the universe. This feature alone eliminates all of the computing required to reach global consensus.

There is no need for universal state. It’s true that many types of data do need to be published to the network — tweets and comments, ride requests, document edits, and so on, to keep with our earlier examples. It’s also true that an app sometimes needs system-wide tracking to monitor aspects of overall activity, just as the body has ways of monitoring and responding to changes in blood oxygen levels overall; this is another reason that the DHT often needs to store some amount of shared data. Unlike on a blockchain, though, each piece of data on the DHT is replicated only enough times to make sure the needed data is always available, including when the author might be offline. We’re talking about maybe dozens of replications in Holochain rather than potentially thousands or more in blockchain. And this limited replication is strategically distributed across all the users participating in the app, which means that each user performs just a little bit of extra work to hold a very small portion of the DHT.

Each DHT contains data for only one application. A blockchain contains all the data from all the applications running on that chain: every Ethereum node, for example, contains all the historical data for all the dApps running on Ethereum. In Holochain, each app has its own shared storage space in the form of its DHT. As Holochain architect Arthur Brock put it recently, “If I just want to run a Twitter-like app, why should I also have to run your crypto exchange, gambling app or collectible cartoon animals? On Holochain, users only run the apps they actually use.”

Each new user to an application adds storage and computing capacity.In blockchain applications, where miners and stakers write and store data, the network capacity is constant no matter how many nodes are added, so increased user activity increases the strain on computing resources. Holochain applications are entirely hosted by the users themselves, so as the demand for the app grows, so does the computing power to run it.

A Truly Peer-to-Peer Network

Let’s use one more example, a social networking application similar to Facebook or Instagram, to summarize the different approaches to data integrity taken by Holochain and blockchain. Let’s also add in the approach that today’s centralized social networks take, as a point of additional comparison. In this social network, you do all the things you’re accustomed to: posting text and images and videos, commenting on other people’s posts, and chatting privately with friends.

• In the centralized scenarios common today, all of the data — including your photos and comments and messages — are held by the company who owns the platform. The social network is supposedly secure and scalable by virtue of its being centralized: the company takes responsibility for maintaining the network, and they are paid to do so in one or more ways. As we have seen, though, our data is often not as secure as these companies might like us to believe: data breaches are extremely common (since data stored in one centralized place makes a honeypot for hackers), plus our data is routinely sold to advertisers or leaked to other third parties (the Cambridge Analytica scandal was but one extreme example).
• A blockchain version of a social network would theoretically be hosted by the miners or stakers running the blockchain nodes. Integrity would be ensured by virtue of your data being written to the blockchain only after a consensus of nodes determines the data to be legitimate, then being replicated across all nodes and stored by them forever. But the data load would be extremely high in this scenario, creating a major scalability problem. Many blockchains and blockchain apps try to solve this problem by reducing the number of nodes that need to reach consensus, or by doing much of the computing or hosting work off-chain on centrally authorized servers. These approaches all compromise on decentralization and point us back, in one way or another, toward scenario #1.
• (In point of fact, even a truly on-chain social network would be only nominally decentralized, since the miners and stakers, who need to be paid to do their job just like a centralized company does, effectively become a new kind of intermediary. But this is the subject of another article.)
• In a Holochain version, the entire network — all of the data and even the application code — is hosted by the users themselves. It’s truly peer-to-peer. Data integrity is ensured through global visibility of local state, established on an as-needed basis. You share your photos, comments, and messages to the network through a shared table called a distributed hash table (DHT), but you remain the primary authority on everything you’ve published. The network is scalable by the fact that no global consensus is necessary, by the fact that the DHT involves only as much data and replication as necessary, and by the fact that every user shares a small piece of the load. The more users join the network, the more capacity it has for scale. And there are no intermediaries at all — no one who needs to be paid or trusted to write and store your data on your behalf.

If a peer-to-peer approach is that much better, why hasn’t everyone been doing it this way all along? One factor is probably the technical complexity involved, but another is probably that it’s difficult for people to imagine that everyone could host their own data and not be able to misrepresent themselves. Even though Holochain has been around in some form for several years, its approach to data integrity is enough of a departure from blockchain’s that developers and users are only just beginning to understand its potential, similar to how it took several years for Ethereum’s capabilities to be widely understood.

That does seem to be changing, however, especially since Holochain’s refactored state model went live and is proving to be many times more performant than previous versions — and also since so many new applications are preparing to launch on Holochain. And we can expect greater and greater awareness of Holochain as more applications go live in the coming months.

At a time when blockchain still has so much potential for growth, it may seem odd to be already talking about a post-blockchain application space. But given the scalability challenges blockchain faces and Holochain’s readiness to leapfrog these issues, it might be time to begin thinking outside the blocks.

One way to stay tuned about Holochain and Holo is to sign up for the occasional newsletter.

Originally published at https://blog.holochain.org on July 22, 2021.

Blockchain: A Holochain Perspective was originally published in Holochain on Medium, where people are continuing the conversation by highlighting and responding to this story.

This has been a good year so far for Cardano. It successfully forked to become a multi-asset blockchain, enabling users to create tokens that run on Cardano natively, similar to how tokens operate on Ethereum. (Full smart-contract functionality remains on their roadmap.) Its token, ADA, became tradeable on Coinbase in mid-March. And investors lined up: the price of ADA rose 560% over Q1, the most among the CoinDesk 20, a core group of cryptocurrencies tracked by the news site. Then in April, Cardano announced a major blockchain-identity initiative in Africa, beginning with a partnership with the Ethiopian government to improve the quality of education.

Cardano makes some promising claims about its platform that seem mainly aimed at differentiating from Ethereum, still the giant in the smart contract space with a market capitalization roughly six times as large as Cardano’s as of this writing. Among its core features:

• Protocols for scalability and security are supposedly more scientific, based on “peer-reviewed academic research”.
• A unique algorithm, called Ouroboros, billed as “the most environmentally sustainable blockchain protocol”. Ourobouros’s consensus system empowers various types of stakeholders, including “input endorsers’’ and “slot leaders”, to mine coins and verify transactions.
• A hierarchical structure that includes a settlement layer for accounting and a control layer for executing smart contracts, supposedly making Cardano ideal as both a medium of exchange and a decentralized world computer.

Whether Cardano is actually better than other blockchain protocols isn’t something we’ll try to settle here. What we can say is that Cardano is seeking to be a better blockchain protocol, which is fundamentally different than what Holochain is doing. Cardano and Holochain may ultimately be solving some of the same problems, but they face very different challenges along the way.

For example, one of blockchain’s biggest challenges is efficiency (or scalability). The computing costs of recording everything that happens on every node of a blockchain are huge, as is the storage load of keeping all of that data forever, so it’s hard to imagine blockchain being suitable for applications involving any kind of big media files. Another issue is latency: blockchains often take several minutes or more to record even simple events, so they’re not very suitable for collaborative applications that rely on fast update times. Users have recently reported transaction wait times on Cardano ranging from 1 to 18 hours.

For these reasons, blockchain projects seem to be primarily setting their sights on financial services applications, which tend to hold relatively little data and don’t require real-time processing. Cardano is no exception, making “interoperability with legacy financial systems” one of its core objectives.

Holochain is not limited in the same ways and, as a result, is capable of enabling virtually all the activities you’re accustomed to doing on the internet now. Holochain is so flexible and scalable because:

Cost: Holochain is completely free to use and run. Because users host themselves and each run a local copy of the application to perform a small amount of the shared workload of validation and storage, there is no need to pay any intermediary miners or stakers. No gas fees also means no built-in token is required to run Holochain apps, eliminating a major usability hurdle. (The tokens people commonly associate with Holochain, Holo Token [HOT] and HoloFuel, actually power the Holo Hosting Network, a peer-to-peer hosting marketplace which enables anyone to participate in a Holochain application through a web browser without having to purchase any cryptocurrency or have a crypto wallet.)

Storage: Holochain is capable of handling large-media applications. Shared data is replicated only enough times to provide a healthy amount of redundancy — perhaps 25 or 50 times, not on every single node as in blockchain — so it is completely feasible to store and serve large documents, images, and videos. Even live video streaming is possible through the shared data environment of a Holochain app.

Speed: Holochain has a number of features that combine to make for near-zero latency:

• You write data to your local chain instantly. Once your data is recorded on your own source chain and then published to the network, the other computers update to your change in their own time — seconds for most computers, perhaps minutes for some — but you typically don’t have to wait for that to happen to continue moving forward. For example, if you’re posting in a social media application, you see your own post immediately, and you can add a comment to it right away.
• In addition, you can send ‘signals’, which are real-time updates transmitted to the specific parties you’re interacting with, ahead of validation by the network. In a chat application, for example, the person you’re chatting to gets a signal so that their screen updates right away with your message, as fast as in any centralized chat application.
• When you are interacting in a way that needs to be validated before you can interact again — such as if you’re transacting with currency — your next transaction only depends on your history being validated by the user you’re transacting with, not by the whole network. (Because all users host a copy of the application, including the app’s validation rules, they don’t need to consult anyone else.) So if there are computers on the network that are slower to update or are offline, that doesn’t slow you down. Incidentally, this feature also makes for infinite transactions per second, increasing with the number of users on the network: if there are a billion people using a currency app, there could be half a billion transactions taking place at the same time — one transaction for every pair of people who are transacting.

For all of these reasons, Holochain is suitable not just for financial services applications but for collaborative applications like:

• Chat applications like Slack or WhatsApp. (Elemental Chat is now live.)
• Project-management systems like Trello and Asana. (Acorn is one example in development.)
• Social media applications similar to Twitter, Facebook, and so on. (Junto is one example.)
• Wikis and other knowledge bases. (H-Wiki and Fractal Wiki are both in development.)
• Calendaring and scheduling applications. (CalendarEvents is one under development.)
• Real-time collaboration tools like Google Docs and HackMD (We created Syn as a simple but powerful protocol for these types of apps.)
• Marketplaces and e-commerce networks for any imaginable good or service, from vacation rentals like Airbnb to ride-sharing networks like Uber to general-purpose marketplaces like eBay or Amazon. One exciting marketplace in development is RedGrid, a peer-to-peer, clean-energy power network.
• Media-sharing platforms like YouTube and Spotify.
• Innovative social organisms like collaborative land stewardship, supply-chain tracking, and novel currency systems.

So what advantages does Cardano have relative to Holochain? The biggest one might be awareness — both of the project itself and of the underlying technology. Cardano has gotten a lot more press than Holochain to date. And Cardano aims to improve on technologies that are already widely understood: distributed ledgers, block production, and consensus-based adoption of global state.

Holochain, by contrast, needs to overcome deep-seated beliefs in order to get developers and users to understand its capabilities. People are so used to the idea that global consensus and universal data replication are the only way to make data secure and immutable that they tend to have trouble believing that it’s possible to guarantee data integrity in a more efficient way. Holochain is a technology at least as disruptive as Ethereum, and remember how many years it took for developers to basically understand Ethereum smart contracts and start building with them? Holochain’s path to awareness is probably more like Ethereum’s in 2014–2017 than Cardano’s today. Then again, more people do seem to be taking notice: the price of Holo Token (HOT) rose even more than ADA over Q1 2020.

The bottom line: Cardano may be positioned to meet a market demand for a better blockchain and give Ethereum a real run for its money. But Holochain may be poised to leapfrog beyond the lot of them, since it is capable of addressing a set of use cases that blockchain protocols simply cannot, with even higher data integrity and the kind of scalability blockchain can only dream of.

Originally published at https://blog.holochain.org on May 26, 2021.

Cardano: A Holochain Perspective was originally published in Holochain on Medium, where people are continuing the conversation by highlighting and responding to this story.

Holochain Dev Pulse 96

You know those words you see on a car’s side mirror that say ‘objects in the mirror are closer than they appear’? In spite of that warning, it’s always a bit surprising when a car comes zooming past, out of the blue, when you thought it was way back in the distance.

I had that sort of experience yesterday morning. During my weekly check-in with Alastair from the Holo hosting dev team, he was practically bouncing in his seat. The reason for his excitement, in his words, was that “Holo is ready”. Alastair normally brings a critical, analytical mind to his work. (He ought to; he’s an engineer.) So when I saw him get excited, I knew there was a good reason for it.

Before you get too excited, though, Holo isn’t going to be in beta tomorrow. He simply meant that it’s finally time to start testing the entire hosting and routing infrastructure.

There have been some big milestones in the past half year. But the next milestone, hosted Elemental Chat, is the one that brings all the pieces together. It depends on:

• The Envoy service and Chaperone browser library (I’ll get to them in a bit),
• Network plumbing to help hosts reliably punch through home firewalls, including a proxy server,
• A registration flow to get hosts onto the network,
• The resolver service, a ‘matchmaker’ that connects web users to hosts,
• Centralised and decentralised routing infrastructure to securely deliver web users’ requests to their chosen hosts,
• An operating system that brings Holochain and the Envoy together on HoloPorts.

Alastair told me that all these foundations are solid now. The routing infrastructure has been in place for a long time, Holochain’s big bugs (including lockups when a host fails to create an app instance for a web user) have been eradicated, the proxy server and gossip protocol are getting some performance boosts, and the weird edge cases involving a user closing their browser too soon have been addressed.

If you’d gotten used to the rapid HoloPort updates over the turn of the new year, like I had, you were probably concerned by the slowdown in the following months. Alastair explained the reason for that too: during our previous release cycles, we involved testers as early as possible. This gave the dev teams a lot of insight into what was and wasn’t working, but it also created a lot of extra work. Our pre-release testers have been absolutely wonderful — they’ve dedicated big chunks of their free time to helping us chase down bugs. But it’s been a lot of work for them, and the communication and coordination have demanded a lot from our small team as well.

So for this release (and the previous minor release), the Holo and Holochain dev teams have put everything through much more rigorous testing before it even reaches internal QA. Once it passes, we expect to have a lot fewer pre-release cycles before it goes out to all HoloPorts — perhaps none at all. And then the fun begins, as all the HoloPort owners invite the web to try out this peer-to-peer chat app for the first time!

We also intend to hire a QA and software delivery manager to streamline this whole process in the future. If you have experience juggling QA, community testers, and deployments, this might be you!

I could tell that Alastair was relaxed about the future of Holo development. With this solid foundation, he said, developing and delivering new features like HoloFuel and the Service Logger app (which tracks resource usage and bills app publishers) is going to get easier and faster.

So what are Envoy and Chaperone?

Envoy is a service that lives on every HoloPort and connects its Holochain runtime to the centralised Holo routing infrastructure and, ultimately, the web user on the other end. It provisions a hApp instance for a user and asks the user’s browser to sign data that Holochain is about to store and publish on their behalf.

Chaperone, on the other hand, is a library that runs in the user’s browser. It handles registration and sign-up, using cryptographic magic to keep the user’s secrets (password and private key) from ever leaving the browser. That means they’ll never get stolen in a massive database hack, and the HoloPort never has the power to do things the user doesn’t want.

The Holo devs have often found themselves working against the centralised assumptions baked into web browsers and the HTTP protocol. Many of the challenges came from security restrictions that keep you safe in the centralised world, but they don’t make sense in a cryptographically secured P2P world where you’re in control of your keys.

In the process of meeting these challenges, the Holo devs came up with a final design that I think is more user-friendly. Previously you had to sign in on every new Holo-hosted page you visit, even if you’d been there before or were already signed in on another browser tab. Now your session stays active across browser tabs and even after quitting and restarting your web browser. This is nothing special for traditional web apps, so it’s great to see it coming to Holo. It means that dApps are finally going to start feeling pretty usable.

If you like technical details, you might want to read this old blog post about Chaperone and Envoy. It was written when the devs were still deep into creating and debugging these components.

What’s next?

Right now we’re in internal QA testing. We expect to invite the pre-release testers into hosted Elemental Chat soon (perhaps by the time you read this) — first as users, then as hosts. We don’t expect to find any big bugs, but we hope we’ll discover edge cases that we hadn’t hit during development. If no critical edge cases show up, we’ll release hosted Elemental Chat to all hosts and invite the web to try it out.

Looking forward to seeing you there sometime soon! (If you have an invite code, of course 😉)

Building with Holochain? We want to hear from you!

Are you building with Holochain? We would love to connect and hear more about your app/platform. Please, take a few moments to fill out this survey! Our goal is to list all the projects building on Holochain on our forum.

Originally published at https://blog.holochain.org on May 7, 2021.

It’s Closer Than It Appears was originally published in Holochain on Medium, where people are continuing the conversation by highlighting and responding to this story.

Holochain Dev Pulse 95

That might be a funny question to ask right now. If you’ve been reading the Dev Pulse for the last couple years, you probably already know what Holochain is.

But for those of you who are new to Holochain, or who want to share it with your friends, we’ve just publicly released a video, What Is Holochain?. It’s based on a talk I gave at this year’s FOSDEM conference, and it gives a quick overview of what Holochain is all about at a level that’ll help devs understand whether it’s a good fit for them.

We’re busy building a new holochain.org website and this video will be featured there. It is fairly technical, but we hope it will build understanding for some folks even if it doesn’t quite get there for everyone. All sorts of people are interested in Holochain — builders, crypto fans, change agents, and business people. And that’s great. But it’s challenging to speak to them all at once! That’s why we’re trying to create materials that focus on specific audiences.

We’re releasing this one first because, ultimately, Holochain is a developer’s toolkit, and developers are the foundation of our adoption strategy. If developers get it, they’ll get excited about it. That means more hApps in the world, which means more hAppy users whose power over their online lives just got a bit stronger.

(Keen eyes will notice that this video has been around for a while. It was actually an unlisted video in my personal YouTube account, but people seemed to love it so much — even non-devs — that we decided it’s good enough to put on the website for now. Expect shorter, more accessible explainer videos in the coming months.)

I know I haven’t shared a lot of updates on Holochain and Holo development lately. That’s partly been because of more important content, and partly because it’s mostly been a continuation of the same work I reported last time. Here’s what’s fresh:

• The new proxy server is finished, support has been integrated into Holochain, and we’ve moved it into QA testing on HoloPorts.
• Gossip is getting some performance tuning to reduce load on the proxy server and make data propagate through the network more quickly.
• The core dev team are changing various components:
• Updated versions of low-level libraries (1.x releases of tokio and wasmer)
• Replacing the LMDB database engine with SQLite
• DeepKey has been ported to Holochain RSM! Its complex data integrity rules have prompted productive discussions about validation, particularly the developer usability of validation callbacks.
• Holochain finally runs on the HoloPort Nano! We’ve been able to compile it for a while, but it crashed on hApp startup. The Holo team did some experiments with wasmer 1.x and a different WASM-to-bytecode compiler, and they’re now demoing Elemental Chat on a Nano test unit. Once Holochain has officially upgraded to wasmer 1.x we’ll have regular OS builds for the Nano, and we’ll begin testing it along with the two larger HoloPorts. (If the hardware is ready before the software, we’ll ship the hardware first and the auto-updater will take care of the software when it’s ready.)
• If you’re using the admin API, there are breaking changes coming to the app installation calls.

Cover photo by Benoit Beaumatin on Unsplash

Originally published at https://blog.holochain.org on April 30, 2021.

What Is Holochain, Anyway? was originally published in Holochain on Medium, where people are continuing the conversation by highlighting and responding to this story.

HoloSign, Anyone?

Recently, DocuSign CEO Daniel Springer said it would never work to create a blockchain version of DocuSign because it would be extremely expensive and inefficient, even though the management of electronic signatures and agreements would seem to be just the kind of use case blockchain promises to address.

He’s totally right. Storing all those PDFs on a blockchain would be ridiculously expensive, because everyone running a blockchain node would need to store a complete record of every document that’s ever been signed. Yikes.

But a decentralized version of DocuSign would be easy to create and simple to run on Holochain. We even thought about building it ourselves just to show how easily it can be done… but then we thought a better idea would be to put it out there to our developer community, since there’s a real opportunity here. If you’ve been wanting a tech business idea for a proven concept with a large addressable market and multiple advantages over the competition, now you have one.

Or maybe DocuSign, HelloSign, or DocHub want to get in on the action? You guys already have the UI and the integrations with other web services, so you’d just be building new data infrastructure, which Holochain makes completely secure and easy to do.

Imagine how elegant this would be:

PDFs are stored not on a centralized cloud server, but hosted by users themselves on their own source chains — locally stored, immutable hash chains containing all the documents they’ve ever signed as well as logs of all the actions they’ve ever taken related to those documents (created, edited, signed, sent to so-and-so, etc.). Users will value owning their own data and having control over their identity and keys — features which are not only native to Holochain’s infrastructure but also guaranteed by Holochain’s Cryptographic Autonomy License.

Users have the choice to install or not install a desktop app — and, later, a mobile app. (Holochain mobile support is in the future plans.)

• If they install the app, they’re self-hosting their data as well as shards of the distributed hash table (DHT) that serves to balance storage across the nodes of the network. The more that your user base chooses to install and run the app, the more they become a part of your hosting infrastructure, which drives down costs. So it will be good to make the app handy and friendly: easy access to all your documents at your fingertips, whether online or offline.
• If they don’t install the app, they sign and access documents through their web browser, the same way that most people use DocuSign and similar products today. Their documents could be hosted on the Holo hosting network, which is a peer-hosting network for serving Holochain apps and data to users not running Holochain apps locally. Web-based users would still be able to download PDFs, just as they can from DocuSign and similar today.

Documents are signed using digital signaturesassociated with user-owned private cryptographic keys (in addition to hand-drawn or script-font signatures), which ensures in a fully traceable way that document signatures are legitimate. The integrity of all data and operations is ensured by Holochain’s peer-validation protocols.

The DHT could ensure that a solid number of encrypted backups exist at all times (maybe 25 or 50) without being endlessly duplicated like in blockchain. This way, no single entity is holding the only copies of the data. (PS — there is probably no need to store documents as PDFs on the DHT; rich text would work just as well and be much lighter.) Alternatively, documents could be stored only on user source chains for an ultra-high degree of privacy not possible in centralized or blockchain scenarios.

Holochain is free to use. Unlike in blockchain, individual nodes are not subject to a heavy computation load, and there is no need to store all the data in each node, so there are no gas costs.

The hosting of documents may also be a cost savings relative to centralized web hosting, depending on the market price of hosting on the Holo Hosting Network.

The bottom line: a Holochain version of DocuSign would be easy to build, would empower users more than existing products, and would be far more feasible than a blockchain version… plus might be less expensive to run even than a centralized version. Electronic agreement management is a good example of a use case that blockchain is ill-equipped to solve, and a great example of a use case that Holochain is perfectly equipped to solve.

If you do take up this challenge, let us know so that we can let the world know what you are up to!

Originally published at https://blog.holochain.org on May 4, 2021.

A Holochain DocuSign Challenge was originally published in Holochain on Medium, where people are continuing the conversation by highlighting and responding to this story.

Holochain Dev Pulse 94

As I pored over the most recent dev updates, I struggled to find a cohesive story to share. There’s no one thing I can point to and say “See, this is what’s happened since the last Dev Pulse.” Just a lot of small updates. And then I realised — that is the message. Right now we’re making consistent, steady progress. Components are being optimised and rewritten, bugs are being found, core apps are seeing accelerated development. For those who would rather not read a tweet stream, here are the major points:

• HPOS, the HoloPort operating system, has seen a version bump of Holochain that’ll make integration of all the upcoming improvements a lot faster. This has involved a bit of work updating the Holo hosting components — Chaperone/Envoy (the parts that connect the user to their chosen host) and the HoloPort/Holochain API.
• One of the upcoming improvements is a new proxy server, which is substantially faster than the previous one. The devs are currently working through a bug in which gossip is sometimes slower than before, and then it’ll go to testing.
• Another improvement is a new database layer built on SQLite. It’s is unoptimised and currently slower than the current one built on LMDB; our expectation is that it will enable optimisations that make it as fast or faster, as well as new Holochain features (such as sharding) that make the entire network more efficient.
• There are now low-level components that install hosted hApps onto HoloPorts (previously the hApps were hard-coded).
• DeepKey, our distributed public key infrastructure application, is in active development. It features the most complex validation code of any hApp we know of, and requires rigorous testing. So we’ve introduced a mocking framework that lets you unit-test any of your functions that use the host API.
• Elemental Chat’s registration process is getting support for joining codes, so we can open it up to the wider world of the web while maintaining a manageable number of alpha testers. This feature will be available to any Holo-hosted hApp that needs ‘membrane proofs’.
• Rigorous testing of the Elemental Chat registration process has revealed a few bugs in the way Holochain handles cell creation.

That last bullet is an interesting one, and I think it might help you understand just a bit more about what’s going on under the hood if I try to explain where those bugs are coming from.

The bugs we’re seeing are all about setting up a ‘cell’ for a new web user on their assigned HoloPort. A cell is a combination of an app binary (or DNA) plus an agent ID, and every agent who wants to use the app must have one running. (In fact, if there are no cells — no users actively using the app — the app can’t really be said to exist.) The cell is how you (or rather, the UI on your computer) interact with your own data in your source chain and others’ data from the DHT.

Preserving your agency

As you probably know, Holo hosting is a compromise between the full P2P, agent-centric world of Holochain and the centralised world of the web. But one thing we refused to compromise was your agency, which is most powerfully expressed in your ability to choose what goes into your source chain. That source chain is a public record of the things you’ve done in the app, and you don’t want anyone else messing around with it.

In order to preserve agency at this level, a HoloPort should never be able to write to your source chain, even if it wanted to. You see, every source chain entry has to be signed by your private key, which is a secret component of your agent ID. The signature has to cryptographically check out against your public key (which appeared in element #3 of your source chain). And that private key is generated and safely stored in your browser every time you log in. That’s what protects you from being impersonated by a rogue HoloPort.

The way Holochain apps work, when you want to perform an action (say, publish a tweet) your UI calls a function in the cell, which ‘stages’ a few writes to the source chain. Then the Holochain runtime validates those writes against the DNA’s validation rules, creates a signature for each of them using your private key, and writes them to storage.

But this creates a bit of a problem if your private key is on your machine and your cell is on a HoloPort. So the HoloPort has to send a signing request back to your browser, consisting of all the source chain writes it wants you to sign. This happens in a separate HTTP cycle.

Now there’s another problem: there’s a very thin window of time, between making a function call and signing the writes, where you might just close your browser and scuttle your HoloPort’s attempt to get those writes approved. If that happens during registration, where your HoloPort is trying to write your Elemental Chat invite code and public key to your source chain (the genesis elements), it can’t actually create a valid cell for you.

So the Holo dev team is working on addressing that edge case. There are two parts to this: one is to fix an issue where the Holochain conductor locks up on failed validation (that’s just a common bug with an engineering solution on the way), and the other is to recover gracefully when the genesis elements can’t get signed on first try.

Keeping bad actors out of the network

With Holochain, every application (or rather, every DNA within every hApp bundle) enjoys its own isolated network with its own separate store of data (its DHT). And sometimes it makes sense to restrict access to that network.

For this we have the ‘membrane proof’, an element near the beginning of your source chain, that supplies some sort of evidence that you should belong to the network you’re trying to join. Depending on the app, it could be a simple invite code, or it could be something complex such as a cryptographic certificate signed by three existing members. This membrane proof is something you supply when you start up your cell, and it’s validated by your initial bootstrap peers before they let you access the rest of the network and its data.

Elemental Chat is one of those applications; it requires an invite code on signup. The Holo single-sign-on dialog has a field you can paste your invite code into; this gets passed on to your assigned HoloPort.

But what happens when you get the invite code wrong? The validator network should reject you before you even get the chance to take a look at the data in the DHT. This is tricky, because you’re supposed to self-validate before joining — that prevents you from publishing bad data. But if validating the invite code requires access to DHT data, you might not be able to self-validate!

This chicken-and-egg problem has sparked a series of discussions about the best way to handle membrane proof validation. It’s an example of the sorts of engineering challenges the dev team face regularly, challenges in which they have to consider the impact on user experience while maintaining commitment to a ‘correct’ design that ensures data integrity. (I love observing their conversations; it makes me feel smart but also gives me a sobering appreciation for how much I still have to learn about Holochain’s inner workings.)

The first step is to fix a bug that locks up Holochain when the Elemental Chat invite code is invalid. As with the previous section, this is just a routine engineering matter. The more important step, currently being worked on in parallel to the bugfix, is to identify the most sensible design for membrane proof validation — one that is easy for developers to understand and use, ensures correctness, and minimises unpleasant user experience.

The most likely candidate design involves a three-step process:

1. The agent’s cell calls a basic sanity check callback on the membrane proof; this can do things like check for correct structure or catch typos.
2. The agent’s cell tries to bootstrap into the network and publish their genesis entries; selected peers validate the membrane proof properly, potentially using DHT data in the process.
3. The agent detects whether they’ve successfully been granted access to the network; if they haven’t, they destroy their cell and try again with a new agent ID (and possibly a new membrane proof, depending on how the app’s membrane protection is designed).

The goal of this process is to catch as many classes of honest mistakes as possible before you try to join the network, while still allowing the immune system to identify truly bad actors and giving you a chance to try again if you’re mistakenly identified as one yourself.

So what does this mean for hosted Elemental chat?

By now you’re probably wondering whether that long-awaited milestone, hosted Elemental chat for web users, is being delayed.

Nope! This is all an expected part of the development cycle. When theory meets the real world, there’s always a process of adjusting the two to match each other. I’m glad to see it happening; it means we’re identifying all the weird edge cases that tend to hide when it’s all just theory. And as we deal with each edge case, it’s just going to make Holo-hosted apps (and Holochain in general) that much better.

Originally published at https://blog.holochain.org on April 16, 2021.

When Signup Goes Wrong was originally published in Holochain on Medium, where people are continuing the conversation by highlighting and responding to this story.

Non-fungible tokens, or NFTs, have suddenly become rather popular. I don’t exactly know why now is their time — maybe it’s because celebrities are beginning to sell them, or maybe it’s because recent price increases of BTC and ETH have given hodlers a lot of new money to play with. Whatever the cause, the world is taking notice. And not just the crypto world — traditional auction house Christie’s recently sold an NFT for $69 million USD.

If you don’t know what an NFT is, it’s a record that tracks ownership of some special digital object, from its creation to its current owner. This record lives on a blockchain or other distributed ledger. The object, on the other hand, lives somewhere else, usually a traditional web server or decentralised file storage system. It might be a piece of digital art, an algorithmically generated cat, an item of digital clothing for your favourite virtual world, or (my favourite, on account of its subtle-yet- undeniable comment on the nature of crypto-trading) a tulip. That can fight other tulips.

The important thing to know is that, unlike normal ‘fungible’ crypto-tokens such as ETH, each NFT represents ownership of something special, unique, one-of-a-kind.

There’s a funny thing about that one-of-a-kind something. Because it’s a digital object, it can be cheaply copied and shared a zillion times, and there’s not much the owner can do about it.

But surprisingly, that’s what makes it so valuable.

If you buy a painting from a famous artist, you can do a few things with it. You can stick it in a vault where nobody can damage (or enjoy) it. Or you can put it in your living room to impress all your dinner party guests with. (As long as they’re in your bubble or you’ve all gotten vaccinated already.)

Or you can put it up in a public place where everyone can enjoy it. Something unexpected is about to happen now: the more people get to see it, the more famous it becomes, which makes it even more valuable. Think about Da Vinci’s Mona Lisa; it’s estimated to be worth $1 billion, and the fact that anybody can print up a copy for their living room hasn’t hurt its value one bit. But the distinction of owning the original — now that’s something special, something that people might be willing to drop a billion on.

Digital art is even easier to copy than the Mona Lisa. So far this has been seen as a threat; music, film, and software companies have fought hard against piracy. That’s because their business models have centred around restricting who can access the things they produce, then charging for access. At the beginning of the internet age, this suddenly became nearly impossible to enforce.

What if we could do for digital art what art galleries did for physical art? What if we decoupled ownership from possession in a way that safely ensured that creators could make a respectable living? What if digital art became more valuable the more freely it was copied and enjoyed? And while we’re at it, could digital tech be used to eliminate (or reduce) the middlemen and let more of the revenue reach the people who do the work to inspire, challenge, and delight us?

According to the website cryptomedia.wtf, this is exactly the sort of opportunity that NFTs hold. Reading this manifesto, I do really wonder if they might just upend the revenue strategy of content producers, allowing them to let go of artificial scarcity and profit from abundance. Could this be an example of a bridge from the extractive economics of the past into a cooperative economics that many of us are hoping for?

In order to do NFTs, we need some sort of registry that records the creation of art pieces, along with subsequent ownership transfers, so people can’t try to sell things they didn’t own. It’d be nice if this registry were publicly auditable — oh, and tamper-resistant, which implies that it shouldn’t be controlled by any particular organisation. And since it’s a piece of software, maybe it could automate ownership transfers and payments too. You can probably see now why blockchain technology is such a good match for NFTs.

Yup!

We talked to a couple of Holochain’s core devs about this, and while they came up with different designs, they all agreed that it wouldn’t be hard to do. You only need one simple rule, a ‘pass-the-ball’ rule, where you only get to pass the NFT to someone else if someone passed it to you (or you created it in the first place). They described this rule as ‘transitive’ — as long as it’s applied to every ball-passing event, it’s guaranteed to be correct all the way back to the object’s creation event.

Some people think you have to add some additional consensus to manage scarcity of NFTs, but in Holochain, every element on every user’s chain is already guaranteed to be unique, and you don’t get scarcer than that. You just need to track the history of who the unique thing has been transferred to.

I find all this easy to understand, because Holochain is modelled on the idea of person-to-person interactions — and we all understand how those work. Holochain adds some extra magic to prevent cheating. I won’t go into much detail, except to say two things:

1. Every time you pass the ball, you need to ask a few people to witness the act. But you don’t get to choose who witnesses it, which helps keep you honest.
2. If someone passes the ball to you, you’re probably going to want to make sure they aren’t cheating. So this NFT app should let you check with a few prior witnesses to make sure the ball wasn’t secretly passed to someone else already.

Plenty of reasons:

• Cost. Because Holochain apps are so resource-efficient, there’s no need to charge a transaction fee.
• Ethics. A lot of smart people have already investigated blockchain’s ecological impact, claiming that we can’t really afford to be adopting this kind of technology right now. A Holochain NFT app, by contrast, would be light enough to run on the basic devices its users already own.
• Developer efficiency. Blockchain NFTs require a couple different stacks for payment, ownership, storage, and UI. With Holochain, those functions could all be integrated into one stack.
• User experience. Not only are crypto wallets really confusing for the average user, but the disconnect between components has caused real losses for NFT art collectors. I don’t think I need to explain why this is a serious problem. Better component integration means a safer — and nicer — experience for everyone.
• Scalability. Holochain apps don’t use global consensus and they run on user devices. That means they scale with the number of people using them.
• Accountability. People have already been selling things they don’t control the rights to. An NFT app on Holochain could leverage some of the identity tools we’re building.

Put together, all of these things could mean greater adoption for NFTs — as well as massive new opportunities unlocked by the efficiency, cheapness, and versatility of a future Holochain-based NFT platform.

Building with Holochain? We want to hear from you!

Are you building with Holochain? We would love to connect and hear more about your app/platform. Please, take a few moments to fill out this survey! Our goal is to list all the projects building on Holochain on our forum.

Originally published at https://blog.holochain.org on April 8, 2021.

NFTs on Holochain? Easy as passing the ball was originally published in Holochain on Medium, where people are continuing the conversation by highlighting and responding to this story.

Holochain Dev Pulse 93

NOTE: Beginning this week, we’re trying something new with the Dev Pulse. We’re going to start publishing smaller pieces more regularly. We think this will make them both fresher and more digestible.

This week we rolled out an update to all HoloPorts. The most visible changes are in the UI of Elemental Chat, our demo app. Users now have ‘identicons’ beside their name; these auto-generated avatars are based on their actual agent ID in the Holochain network and can’t be forged. This is important because usernames are just labels; you can set yours to anything you like, even someone else’s name. If you’re talking to a friend and suddenly their identicon looks slightly different, there’s a good chance someone is trying to impersonate your friend.

This is an example of how cryptographically-secured data is a good first step to ensuring authenticity and accountability without the need for a central authority.

There are some under-the-hood changes to Elemental Chat as well. The network stats are calculated more accurately (you can see them by clicking that graph icon in the upper-right corner). And the real-time performance of chatting (using signals) has been tweaked to be more efficient; this should result in a small performance improvement.

Under the hood, the HoloPort’s operating system is using a new version of Holochain RSM. The reason this matters is that it prepares us for the next infrastructure milestone: hosted hApps! The speedups so far are modest, but we expect that other near-term updates will make Holochain substantially faster, fast enough to support thousands of users per app.

These future updates will include:

• A switch to a new storage engine based on SQLite, the world’s most popular database engine. I know, I know, last year we said we were switching to LMDB because it was the best thing ever. And in comparable tests (hash table lookups) it does outperform SQLite. But we need more than just hash table lookups — we need fast querying with arbitrary filter parameters. This is especially important if we want to do neighbourhood sharding well. And sharding will lift the lid on scaling, allowing hApp networks to grow far beyond what traditional blockchains can reasonably handle.
• The proxy server, which helps HoloPorts and Holochain users punch through home firewalls, is in the midst of a large rewrite. So far, benchmarks predict that it’ll be 5× faster for raw throughput, at the very least. Once this is deployed, it will enable us to see places where DHT gossip and validation performance need to be improved.

For developers, all of the above means that Holo hosting is on its way. The Holo Web SDK is being readied for public use as well, and we’re exploring the possibility of offering automated hApp testing services in a real hosting network on real HoloPorts.

Special note for hApp devs

The develop branch of Holochain has advanced quite a bit past main in the past month or two, bringing features like the hc dev tool and hApp bundles. For that reason, I'm now recommending that you use https://nightly.holochain.love to install/enter the development shell. Also, take note that we might be changing installation URLs in the future, but we'll always keep the installation guide up to date for you.

Cover photo by Paweł Czerwiński on Unsplash

Originally published at https://blog.holochain.org on April 5, 2021.

Elemental Chat Gets Freshened Up was originally published in Holochain on Medium, where people are continuing the conversation by highlighting and responding to this story.

Holochain Dev Pulse 92

Summary

Mostly we’re working on performance, bugfixes, and features (proxy server, storage code, sharding, and validation receipts), but we’re also working on a few developer tools: the hc command (yes, it's coming back), a mock HDK, and a nearly finalised hApp bundle spec.

Highlights

• The hc developer command returns!
• Mock HDK: Test your DNA without firing up Holochain
• hApp bundle spec ready for basic use

The hc developer command returns!

Veteran Holochain developers will remember the hc command, a sort of Swiss army knife tool that let you scaffold, compile, test, and run your hApps. It's here again and (almost) ready for you to start using in your daily development.

Subcommands already implemented:

• hc dna replaces the existing dna-util command.
• hc dna init creates an empty directory and manifest file for a DNA.
• hc dna pack packs a collection of compiled WASM zomes and a DNA manifest file into a DNA file. It replaces dna-util -c.
• hc dna unpack unpacks a DNA file into its component manifest and compiled WASM zomes. It replaces dna-util -e.
• hc app has the same subcommands as hc dna, but works at the level of a hApp bundle - that is, a collection of DNAs. (See below for more info on bundles.)
• hc sandbox is still a work-in-progress, but will let you generate and run conductor 'sandboxes' - temporary conductor configurations (including multiple instances on multiple conductors) that let you test-run your application.

This tool isn’t available in mainline Holonix quite just yet, but you can try it out with this command:

nix-shell https://github.com/holochain/holonix/archive/pr/bump-holo-nixpkgs-add-hc.tar.gz

Once it’s finished its work and you find yourself in the development shell, you should have the hc tool available:

hc --version holochain_cli 0.1.0

We’re still working out what scaffolding should look like beyond the simple hc dna init and hc app init commands. If you have an idea of what you need from a scaffolding tool, please create a GitHub feature request and tag it with 'enhancement'.

What about scaffolding and testing? Testing is already well-covered by the tools native to each language:

• You can use npm test to run scenario/end-to-end tests written with Tryorama
• You can use cargo test to run unit tests with the new mock HDK (see below)

Mock HDK: Test your DNA without firing up Holochain

Unit testing — tests for the smallest executable pieces of your code — should be the foundation of your testing plan. Up until now, though, we haven’t had any great tools for unit testing a DNA.

Tryorama, our scenario testing framework, has been great for doing integration testing. It loads up an entire conductor (or multiple conductors) to host your DNAs. But this isn’t what you want for unit tests, which are meant to test a specific piece in isolation. It also doesn’t support a lot of edge cases — unexpected conductor shutdowns are easy to model, but other situations like hardware errors and malicious peers are impossible.

So the core devs have introduced a mock HDK that lets you simulate host call responses and introduce all the edge cases you like. You can find examples throughout the WASM test suite; here’s one that mocks the CRUD host functions.

hApp bundle spec ready for basic use

We expect a common pattern for hApp development will be mixing-and-matching existing DNAs in new configurations, sort of like an ecosystem of microservices. In fact, we explicitly want to see that happening. We think Holochain will get adopted more quickly if developers can lean on each other’s good work; it saves work and time for developers, and leads to a better user experience when new apps can connect to existing apps and take advantage of users’ existing data.

To that end, we’ve long held an intention to let developers easily specify a ‘bundle’ of DNAs that make up a full application. There are different needs; here are a few:

• Connect to an existing DNA, such as a core app (like DeepKey) or other popular app (like chat or social media)
• Make a new copy of an existing DNA so users can enjoy a private space using the same code
• Clone a DNA multiple times for separate, small spaces — such as team chat rooms within an organisation

hApp bundles existed in Holochain Redux; now they’re coming to RSM. A few of the above scenarios (e.g., cloning, DNA versioning) aren’t supported yet, and the bundle spec format isn’t officially stable. But it’s already in use in the admin API (and soon Tryorama and the ), and the parts that hApp devs are already using look pretty robust and probably won’t change much. Here’s what works and what’s changed:

• The admin API endpoint InstallAppBundle is meant to replace InstallApp, which will be deprecated. It supports all the features of InstallApp already and will support more in the future.
• DNA manifests, which already exist and can override properties and UUID in the DNA bundle. DNA manifests have only had a minor tweak to allow referencing DNA bundle files by URL as well as file path.
• An app bundle has ‘slots’, which give a human-readable ID to existing or new DNAs in a bundle. In the future, these slots will also be used for cloned DNAs.
• In contrast to Redux’s bundle spec, DNAs don’t explicitly depend on each other via bridge specs. The cell can simply make a bridge call to any other cell it has a handle for. Access is granted or denied via capability-based security. (This means you have to be careful about circular dependencies.)

Bonus: signs of ecosystem growth

I’ve noticed some things lately that signal to me that Holochain is gaining awareness and support in the open-source developer community.

• hApp developers are creating pull requests that add features — both ones that meet their own needs, like being able to specify header timestamps, as well as ones that everyone can enjoy, like mDNS peer discovery for local networks.
• People from outside the Holochain dev community are offering their own gifts, such as in this forum post introducing a low-code development toolkit (soon to be open-sourced) and inviting collaboration to make it work with Holochain, as well as this post investigating Holochain’s usefulness for a popular Kenyan community currency.
• Engagement on the developers’ forum has seen a steady climb in the past two months. If I could guess, I’d say this is partly due to the fact that people can now actually see Holochain in action, thanks to Elemental Chat landing on HoloPorts, and partly due to the work we’ve been doing to increase awareness at events like the FOSDEM 2021 and Hello Decentralization conferences.

Connect with us!

If you’re a developer and you’re just learning about Holochain, I encourage you to join the developer forum and talk to the other friendly community members. If you’ve got an application (or an idea for an application) and are wondering how it would work with Holochain as a back end, I love talking about the nitty-gritty details of implementation (my username is @pauldaoust). The community also runs courses; there’s an application architecture course currently running and it’s likely to be offered again in the future.

Originally published at https://blog.holochain.org on March 13, 2021.

Dev Tools in the Works was originally published in Holochain on Medium, where people are continuing the conversation by highlighting and responding to this story.

Holochain Dev Pulse 91

Summary

The big news is that the core team’s work on the HDK, Holochain’s standard library for building DNAs, has slowed down and shouldn’t see any big changes for a while. Work is now shifting back to core features, such as sharding the DHT.

In the meantime, the Holo team is getting Elemental Chat ready for Holo hosting. They’ve also enabled the Host Console on all HoloPorts (read on to find out why).

Highlights

• HDK stable for now, gets version number, available on crates.io
• Holo progress: hosted hApps and Host Console

HDK stable for now, gets version number, available on crates.io

To those brave application developers who took the plunge and started migrating their apps to Holochain RSM (or starting fresh on RSM without prior experience), I applaud you for your perseverance. You’ve held on through lots of uncertainty — little breaking changes, rough edges, and in some cases total rewrites.

I’m also happy to tell you that there’s light at the end of the tunnel: thanks to steady feedback from application developers, we’ve managed to get the HDK into a fairly stable state and we intend to leave it alone for a while.

You can read a summary of all the most recent changes on the Holochain Open Dev blog — there’s a lot of good stuff in there.

This means we’re giving it a version number — the first since v0.0.52-alpha2, the last release of Holochain Redux and its HDK. It’s going to be called v0.0.100, and it’s likely that the HDK’s version number will eventually be decoupled from the Holochain runtime’s version number so that they can be updated separately.

And it also means that we’re going to be releasing the HDK on crates.io once again — probably by the time you read this. This means no more messing with long GitHub strings in your Cargo.toml file!

You’ll also have up-to-date online documentation at https://docs.rs/hdk/ . Speaking of which, part of the most recent sweep of the HDK was to get its documentation up to a high quality level. Take a read, let me know how it is for you, or even maybe fork the repo and contribute your own changes if you feel so inclined ;-)

Holo progress: hosted hApps and Host Console

The Holo hosting team has been putting most of their work into the login process and the Holo Web SDK, a JavaScript library for integrating a web app with the Holo hosting network. The register/login form has been built and tested (I tried it out myself last week but didn’t think to grab a screenshot, sorry). The current step is to get signals — the real-time node-to-UI messaging feature — working with Holo. HoloPort owners who’ve already been using Elemental Chat will be familiar with this feature; it’s what makes it feel as fast as a cloud-based chat app.

I just learned that they also enabled the Host Console on all HoloPorts. There’s not much to see — yet — but what is there promises a pleasant user experience. The big reason for rolling this out now, before hApp hosting is enabled, is the ‘Access for HoloPort support (SSH)’ switch. This lets you open up your HoloPort to a tiny vetted group of team members so they can directly diagnose problems in this alpha phase. We recommend that you don’t turn it on unless a support agent from Holo asks you to. And remember that, in the spirit of agency, you can turn it off any time you like.

That’s it for now. If you’re interested in getting into Holochain app development, the team and community are producing lots of content — check out the Build It! series with David and Philip, or Low Code Zone for a more philosophical approach. We’ve also been appearing at online conferences lately, which has resulted in some good video and text content for builders — we’ll share those in the coming weeks. And of course you can visit us on the developers’ forum, where exciting conversations emerge and merge regularly.

(cover photo by A lo Bonzo on Unsplash)

HDK is stable (for now) was originally published in Holochain on Medium, where people are continuing the conversation by highlighting and responding to this story.