Our introduction to trend-following posts on Zerodha Varsity (Part I, II, III, IV and V) used tradfi instruments to build a basic model. What if we applied the same principles on crypto assets?
To keep things simple, we’ll pick only two assets: BTC and ETH. The portfolio is evenly split between the two. Since crypto markets are 24/7/365, we’ll divide each day into 24 hour slots and construct a daily series based on the closing prices at each hour. The portfolio is further split into 24 parts each. Each position is an average of a binary trend signal.
Individually, trend-following boosts the Sharpe ratio of each asset.
You may not have captured the absolute highs but you would have avoided the steep drawdowns.
They are stronger together than individually.
Needless to say, leverage in this scenario would be fatal.
Hyperliquid is probably the most hyped DEX (decentralized exchange) right now. The advantage the CEXs (centralized exchanges) had was in the “ease-of-use” area, especially in the on-boarding and trading APIs front. Hyperliquid tries to marry the convenience of a CEX and the benefits of a DEX.
We downloaded their historical order book to check the spreads and depth of popular coins.
BTC, ETH and SOL are a world apart – tight spreads and deep books.
However, trading the less popular coins needs some care.
Zooming into the spreads brings out the difference between the top-tier coins (BTC) and the rest (PENDLE).
The ranges look decent if you are trading a retail-sized book.
“Zero Knowledge”, contrary to what it sounds like, is actually quite interesting and fun. It might even be a solution to our long standing problem of validating the world’s transactions without a trusted third party or government or central bank. If you Google for the terms Zero Knowledge and Blockchains, you will be flooded with whitepapers, articles, explainers, investment advice, and everything in between.
What does Zero Knowledge (ZK) even mean? Let me start with a toy example, and then we can work our way up to world peace.
Say we both get the same newspaper and it has a Sudoku puzzle in the games page. I claim to you that I know the solution to this puzzle, but will not tell you what the solution is. Being the frenemy that you are, you won’t believe me, obviously. Can I prove it to you, beyond reasonable doubt, that I do know the solution to this Sudoku puzzle, without telling you what the solution is? More formally,
If I know a solution, I should be able to convince you of that. Without leaking any knowledge about the solution.
If I lie about knowing the solution, I should be caught – with overwhelming probability.
If both the above are possible, that would be a Zero Knowledge Proof of Knowledge of a Sudoku puzzle. There are some ingenious ways of doing this, which rely heavily on cryptographic primitives and protocol design. In fact, it’s possible to convince an audience that you know the solution for almost any puzzle without giving them any hint of the solution itself. Sudoku was just one example. You could prove that you know the solution to a crossword puzzle, or the Rubick’s cube, or that you know a cycling route from New York to Seattle that’s exactly 5000km, or that you have paid your rent, or that your bank balance is more than $10,000 or any such statement really – without actually revealing the actual solution to the statement.
Imagine the power of such a system, where you could convince others that something is true, without revealing how it is true. In most real world systems, including financial systems, to prove something to someone, you have to reveal the actual facts of the matter – and thereby reveal more than you have to.
For example, getting a visa to any country requires you to provide your bank statement – just to prove that you can afford the trip. It should be possible to prove that you can afford the trip, without revealing any financial information. Also, the proof should be real – as in, if you cannot afford the trip, you shouldn’t be able to prove such a thing and fool the visa-issuing agency. We want both sides, the prover and the verifier, to win. Just with no leak of extra information. The best kind of privacy, if you will.
Where is Waldo?
First, I will give an example of how such a Zero Knowledge protocol looks like, to make you believe that it’s possible. Below is Waldo: Say Hi to him.
Waldo is somewhere in the amusement park image below. Can you find him? Don’t try too hard, it’s not worth it.
This “Where is Waldo” puzzle lends itself very well to a Zero Knowledge protocol. I can prove it to you that I know where Waldo is without revealing his actual location on the image. How do I do that? We run the following protocol between the two of us.
You blindfold yourself. I keep a large white sheet of paper on top of the amusement park image, and ask you to remove your blindfold. You can give me one of two challenges. You should choose these challenges randomly.
Remove the sheet of paper and show you the amusement park image underneath.
Cut out a small hole in the white paper right above where Waldo is on the image. If I do this, I must know where Waldo is.
Repeat step #1 till you are satisfied.
Why does this protocol work?
If I know where Waldo is, I can easily answer challenge #2. That part is easy. It’s not so easy to figure out why challenge #1 is required.
I could cheat by keeping some other image under the paper which has just many images of Waldo on it. How do you know that it’s actually the amusement park image and not some other image that I made up? Challenge #1 to the rescue. If you had asked me challenge #1, I had to remove the entire paper and show you that this was the amusement park image in question.
Note that you cannot give me both the challenges at the same time, as that would tell you where Waldo is. Only one challenge per protocol round.
If we do this entire exercise just once, you could have asked me to answer challenge #2 and I could still cheat with a probability of 50%. If we do it twice successfully, I can still cheat with a probability of 25%. If we do it three times, it reduces to 12.5%. If we do it 10 times, and you picked your challenge randomly each time, I can cheat only with a probability of 0.1%. If we repeat this 20 times, the cheating probability drops to 0.0001%. And so forth, exponentially. Again, this only works if you pick your challenge randomly. If I know in advance that you will ask me the challenge sequence, of say, 122212121222111 – I can pass all challenges easily. The protocol works only if I am unable to guess your challenge sequence.
Cryptographic researchers have proven that almost any statement can be proven in zero knowledge. Imagine that! Any statement! It’s one of the most celebrated results in theoretical computer science, all the way back from 1986. The concept of Zero Knowledge Proof itself was introduced in 1985, after the original paper was rejected in major scientific conferences in the prior years because of how absurd the idea sounded. It still sounds counter-intuitive, if you ask me.
One popularly used ZK-proof system, solving a very specific problem, is that of Digital Signatures. When you digitally sign a document, you are proving to the verifier that you know a secret key to your public key (which the verifier already knows, or is tied to your identity, or some such). For the longest time, general purpose ZK-systems, which could prove any statement, were just theoretical results – the actual proofs themselves can be quite unwieldy and inefficient. Theoretical work continued, but there were still no practical applications that needed these proofs to get smaller, or easier to understand, or even remotely workable. 25 years went by, and people were mostly happy with either revealing everything about something to prove it, or having a trusted third party (like a Bank Officer or Notary) signing a statement saying that something is true, without revealing the underlying details. Ho-hum.
Enter Bitcoin!
Bitcoin removed the trusted third party from financial transactions. Or at least, introduced the idea that it could be done with clever cryptography and protocol design. Researchers who were toiling away in obscure labs and universities were suddenly like: “Hey, there are these amazing theoretical cryptography results from decades ago, let’s use them”. These ideas suddenly seemed ripe for more R&D to make them practical. And boy did the researchers and engineers deliver! Here’s a short list of how Zero Knowledge pervades the cryptocurrency space.
New cryptocurrencies: Zcash, Monero, Grin, Beam, Mina, etc.
Everything about a transaction is hidden. Who is paying. Who is the recipient. What is the amount. Everything is hidden. Crucially though, verifiers can verify that the transaction is valid, and no one is cheating anyone. Zero knowledge magic. Details differ, but this is the general idea.
Additionally, Zero Knowledge proofs can verify large numbers of transactions without needing to store all those transactions. So, these ZK-blockchains can be as small as a few KB. For comparison the Bitcoin blockchain is 350GB and growing. Ethereum’s blockchain is 1TB or 5TB (depending on whom you ask) and growing.
Layer-2: ZK-Sync, StarkNet, etc. bring the benefits of ZK-proofs to legacy blockchains like Ethereum and increase throughput quite dramatically.
Other Proofs: Exchanges can use ZK-proofs to convince their users that they are not doing fractional reserve or rehypothecation shenanigans, and in fact, do custody all their customer assets.
What next?
Some of these general purpose ZK-systems have quite advanced cryptography, and their security guarantees are proven sometimes under ideal settings. When I say security guarantees, what I mean is:
Can the prover cheat?
Can the verifier learn something by violating the zero knowledge principle?
Can we do the entire thing without relying on cryptographic assumptions?
Some systems rely on an initial ceremony where some trusted party has to do one-off computation. Can we remove such requirements?
Practical minded people say that this stuff is too advanced, or “moon-math” as they call it. These primitives will not make it to Bitcoin for a LONG LONG time, if at all. Bitcoin’s cryptography is from an even older generation, and has been vetted in traditional settings like e-commerce, national defense, etc. No moon-math for Bitcoin!
That doesn’t mean that Bitcoin won’t benefit from these new developments. Bitcoin has evolved to a place now where the core protocol itself won’t change that easily, but additional features have to be built on top, in other layers. ZK-proofs will reside on a secondary layer somewhere on top.
Ethereum, on the other hand, is more open to these ideas. ZK-proofs are making their way into Ethereum’s core-system slowly, but will definitely pervade Ethereum’s Layer-2 ecosystem quite thoroughly in the near future. Much faster than in Bitcoin, from what I can see. Newer blockchains will go all-in, and will be built around ZK-ideas, or will offer them as native operators or subroutines.
You have the entire spectrum of blockchain platforms – some boringly conservative, and just trying to be sound money. Some others on the bleeding edge of maths, offering true privacy through ZK-proofs and the like. I expect these to become more mainstream as privacy becomes non-negotiable. Currencies, smart contract platforms, exchanges, and every other financial intermediary will go maths-first!
or: How I Learned to Stop Worrying and Love Nation State Attacks.
Bitcoin’s secret sauce, and how it works, was on full display these last few weeks. Bitcoin was designed to work against the most powerful of adversaries, and boy – did the adversary show up!
The China Syndrome
A few months ago, 45% to 75% of Bitcoin mining happened inside China. Then the Chinese government banned it.
There are anecdotal accounts from people on the ground are seeing Bitcoin mining operations being shut down by law enforcement agents. And there are similar accounts from people on the ground elsewhere in the world where containers full of mining hardware are being shipped to, lock, stock and barrel.
And then there is the Bitcoin blockchain – the source of absolute truth.
I have a copy of the Bitcoin blockchain on my computer, and could actually run the numbers myself and see that the production of Bitcoin blocks slowed down dramatically. Here’s a plot of how long it took, on average, to find 2016 blocks from 12-May-2014 to 18-July-2021.
Bitcoin blocks, on an average, are supposed to be generated once every 600 seconds. But you can see the spike in this number on the graph towards the end, going all the way up to 832 seconds. This means that during that period, the total number of active miners went down dramatically, and that led to the inter-block average-gap increasing equally dramatically from 600 seconds to 832 seconds.
Putting the anecdotal and canonical sources of data together, we can be reasonably certain that the Chinese mining ban lead to a global drop in Bitcoin mining.
Does it matter?
Not really. Miners come, miners go – Bitcoin chugs along. That is what it is designed to do. Bitcoin targets a block production rate of 600 seconds per block. If Bitcoin’s design had been naïve, whenever its dollar value went up, more miners would enter the system to make more money, and blocks would arrive faster than 600 seconds. Similarly, if its value went down (or if governments kicked them out), miners would leave the system, and blocks would arrive much slower than 600 seconds. The block production rate on either side of 600 would persist, and reflect the total number of miners in the system.
But no, that’s not what happens. No matter how many miners are in the system, it always takes around 600 seconds to mine a block. This is done through the difficulty adjustment algorithm, also known as Satoshi’s stroke of genius.
Before we get to the difficulty adjustment algorithm, we have to first understand why keeping the inter-block interval of 600 seconds is important. Bitcoin works because everyone can check whether their perceived ownership of their own Bitcoin is fact or fiction. To check this, you need access to Bitcoin’s data? Where is this data? How big is it? How do I access it? Bitcoin’s data is not held by some central custodian, or a bank. It’s held by everyone who is interested. It includes all transaction from the genesis block onwards – from January 2009. But storing everything with everyone sounds crazy – and to be honest, it is crazy. But the more you think about it, the more you realize that there are no other easier ways of doing self-validation, other than offloading the “do I control my money or not?” question to someone else – and trusting them. Bitcoin prefers the opposite: self-validation.
So, if we accept the crazy idea that everyone stores a copy of the blockchain, we have a fundamental tradeoff – the blockchain cannot get very big (by growing very fast). It also cannot stay static: new transactions need to be added every so often to facilitate economic activity. Currently, the blockchain is around 377 GB, and growing at around 50 GB per year. If it grows too fast, not everyone will be able to hold their own copy. If it doesn’t grow fast enough, there is not enough transaction space to accommodate the demand for transactions. Under these constraints, Satoshi decided that a 1MB block every 10 minutes is a good tradeoff. To keep this tradeoff constant, blocks cannot be generated slower or faster.
What happens if Bitcoin’s value skyrockets and everyone wants to be a miner? Remember that a miner who generates a new block gets to keep the newly minted Bitcoin that comes out of each block. So, if the value of Bitcoin goes up, expect more miners to materialize. To accommodate this, Satoshi designed a simple algorithm that makes mining harder or easier depending on how long it takes to generate the previous 2016 blocks.
The Bitcoin protocol contains a positive number called “difficulty”, whose value is currently 13,672,594,272,814. This number controls how hard or easy it is to mine a block. Let’s say the total time taken to mine the previous 2016 blocks was greater than 2016 times 600 seconds, by a factor of X. This difficulty number is then adjusted lower by the same factor X. If the time taken to mine the previous 2016 blocks was lower, the difficulty number is adjusted upwards – again by the factor X. That’s it.
As far as “algorithms” go, this is as simple as it gets. It’s middle school level arithmetic. Other than combining existing ideas from cryptography and distributed systems, Satoshi’s only novel contribution was this middle school level formula. The genius, as they say, is in the simplicity of it.
When these erstwhile Chinese miners turned down their mining hardware around end of June/beginning of July 2021, Bitcoin’s mining difficulty dropped from 19 trillion to 14 trillion, by around 5 trillion – which is around 28%. The reduced difficulty made it easier for the remaining online Bitcoin miners to start generating blocks every 10 minutes again. The next 2016 block average was 630 seconds. Voila!
As Bitcoin’s value increased from 0 to wherever it is today, miners have only entered the system – and have rarely left. Difficulty has always gone up – to accommodate this increase in value. So, how does this difficulty number actually make it easier or harder to mine a Bitcoin block?
The Proof of Work Function
Bitcoin, famously, relies the “partial hash-preimage puzzle” to build its Proof of Work function:
You double hash data from the block you want to generate, and check if that hash value is less than the target on the right hand side of the equation. If it’s not, you change the block data, and try again, and again, and again, and again.
For example, if I double hash make-believe block-data, say the string “Bitcoin forever!”, I get the number:
So, it doesn’t work. I need to keep trying the function again and again with different block-data to hit gold. The actual previous Bitcoin block’s hash was 888160945014446794317532755205888398236464272495427689, which is under the required target, and that miner struck gold – so to speak.
If the difficulty number goes up, the mining target goes down, and finding block-data that double-hashes to a number lower than that target gets harder. It’s like tossing a 6 sided dice and wanting to hit a number less than or equal to 1. It happens only once every 6 times. If difficulty were to reduce, the target would move to a number less than or equal to 2. That happens every 3 times – mining just got easier.
Why go into the nitty gritty details of this function, with all the associated arithmetic and probability? I want to get into the properties that this unique function has, that makes it ideal for Bitcoin mining – and resisting nation state attacks.
Parameterizability: The function provides very fine degree of control over how much harder or easier we want the function evaluation to be. If you increase or decrease the difficulty number, the function becomes easier or harder to evaluate, respectively.
Memorylessness or Progress-free ness: Even if you have already run the function a million times, it still doesn’t give you any advantage over the next run. Each run of the function is what is called a Bernoulli trial – with the odds of hitting gold the same no matter how many times you have tried in the past. This makes sure that larger miners have no other advantage than just the larger chance of producing a block. If this property weren’t there, the largest miner would *always* win, even if they had just 0.0001% more power than the next largest miner.
The other incredible advantage of Memorylessness is that a miner can be turned off, put in a container, shipped elsewhere and plugged back in. The only loss the miner incurs is the Bitcoin that could have been mined in that interim time when the machine was turned off. Most physical objects being built, or even computations that are being performed on computers rely on previous data or “progress” that has been done, stored and retrieved, so that we can continue the process further. Shutting down something abruptly, without needing to store any state of progress, and starting elsewhere without any extraneous loss is not that common. This allows Bitcoin miners to be incredibly mobile and seek out the cheapest electricity wherever it exists. They are, in the true sense, plug-and-play.
Hard to compute, but easy to verify: To get the double-hash value which is under the target needs millions of trials of the function. But once someone finds it, the rest of us can verify it immediately with just a single iteration of the function. This, again, makes decentralization possible – where all of us can run the Bitcoin software on our computers and check that the miners are doing the right thing.
Replacing this function is not that easy. Most attempts have kept the general idea, and have tinkered with the specifics.
Conclusion
A nation state the size of China attacked Bitcoin where it’s supposed to hurt: Bitcoin Mining and all they managed to get in return was a giant shrug of indifference by the protocol. Yet another instance of Bitcoin living up to its promise of being designed to last forever. This self-adjusting nature of Bitcoin – that makes it change itself based on market conditions, with no one central entity being in charge – separates it from all other forms of money. Fiat money always has a central planner. Bitcoin has a protocol.
