**A** (0:07):
Hey everyone. Nice to meet you guys. Thank you guys for coming to the validators panel. My name is Sunny Agarwal and I'm currently a core developer and researcher on the Cosmos project. So we've been doing a lot, I've done a lot of like the architecture and design for the staking side of a lot of what Cosmos is doing. So you know, I've been thinking a lot from like about like what validators want to do and whatnot. And so now it'd be great for me to like hear from the validators themselves. So would the rest of you like to introduce yourselves?
**B** (0:40):
Why don't you start with.
**A** (0:41):
Yeah, can you start up?
**C** (0:42):
Sure. I'm Tim Ogilvy. I run Staked. We run validator nodes on behalf of typically institutional investors. Brand name funds we run across. Currently we're live on four chains and we'll be live by roughly 10 chains by the end of the year.
**D** (0:59):
Hey guys, my name is Joe, one of the founding members of E on Staking Partners. We are building a staking on ramp for retail institutional investors and supporting network infrastructure along the way.
**E** (1:11):
Hey, I'm Adrian. I'm one of the co founders of Crypt Gym Labs, a validator spawned out of the Cosmos project. Really? So some of you may know me for my work in Cosmos. And so that's sort of how this all started because we needed validators for Cosmos and I very much got involved with how we can run validators for Cosmos. Currently Cryptium Labs is live on Tezos and running testnets for Cosmos. Polkadot. Yeah, that's it.
**B** (1:40):
My name's Zaki Manion. I do sort of run R and D at Tendermint that's creating Cosmos and I also have a validator called Occlusion that I run with a partner. We're live on the Cosmos test nets and probably going to add some more networks in the future.
**A** (1:57):
Cool. And also just for context, I'm also running a validator called Sitka. But I'll try to remain neutral during the panel as a moderator. Okay, so let's just start off right off the bat. Like what even are validators? Are they specifically something for proof of stake or is it a more broad term? There's this terminology called like masternodes that has been around for like years. What's like the relationship? Is it essentially close to the same thing? How are they different? Yeah, whoever would like to take that?
**E** (2:30):
Right, so like validators, it's like when I think about validator, I think about like the only problems that a validator is solving is secure and available cryptographic keys to provide digital signatures. Like that's the entire thing of validator is right, because you have to keep some keys that are used for consensus signing. They are usually very applicable to proof of stake networks and sort of fulfill the security functions of proof of stake networks. But there's nothing to say that validators couldn't also become a term that sort of describes trusted oracles, like where they start providing, let's say, authenticated weather data into numerous chains. Because it's essentially fundamentally the same problem of available and secure cryptographic keys. So that's what I think about validators.
**B** (3:15):
So sort of the emergence of validators is sort of this creation of a new role in the cryptographic economy. So like we've had service providers like exchanges, we've had miners, we've had wallet providers. Validators are kind of like a new piece of the puzzle. And so like Adrian described, like what is unique about a validator, which is a secure and available cryptographic key? So given a secure available cryptographic key, like what does that infrastructure look like? It looks like the infrastructure for higher throughput networks. It looks like the infrastructure for how do you have distributed ledgers that don't involve mining all of those pieces. It looks like oracles. And so I think that the reason why all of us are interested in this business is it's sort of this emerging opportunity that we have these networks that, that now support this functionality and it's important and like we're sort of interested in realizing sort of the economic opportunity on top of it.
**A** (4:14):
So what do you say that like the economic opportunity on like getting in on staking now, like we're just seeing this year we see a lot of proof of stake blockchains emerging. Would it be like similar to being like a miner in the early days of Bitcoin, like just jumping right into that?
**C** (4:29):
Yeah, I think the big difference between mining and staking is that mining is external to the network. And so some external miner is going to create new blocks. With staking, you, if you hold a proof of stake currency, can earn effectively an interest rate by voting for valid validators and people who are going to do that job correctly. And so it creates a real economic opportunity for not just the folks on the stage, but everyone in this room. To the extent you hold something, you can earn an interest rate on it by choosing wisely.
**D** (5:04):
Yeah, just to touch on that, I totally agree. It sort of democratizes security for everyone and the inflation rate gets to be distributed in a more equitable manner amongst all the stakeholders in the network, rather than just, you know, a couple entities that are like consolidating hash power and colluding. So, you know, there will be issues like that down the line, obviously, but I think, you know, having the crowd involved is important.
**B** (5:29):
So the way I think about it is there's an obvious supply and demand mismatch where there are a lot of networks that are launching that are sort of creating space for tens of thousands, for thousands or tens of thousands of validators, and there's maybe a hundred credible validator operations in the world right now. So there's that supply, demand mismatch creates a significant opportunity also.
**E** (5:53):
So in with proof of work mining, what you always had to think about, you had to bet on the underlying success of the network, right? Like if you wanted to mine Bitcoin, you had to bet on the success of bitcoin itself. With validation, this isn't necessarily true. As a validator, you aren't necessarily betting on a single chain to succeed, you're rather betting on the fact that proof of stake as a whole works, right? Because to you it's essentially almost, it's not exactly zero cost thing, but it's like validating a second network or third network is essentially zero cost to you. So instead of having to bet unless you need to purchase the underlying staking token, but since most networks have delegation, you can literally run a validation service that isn't exposed to the volatility of the underlying assets, which is sort of nice because at that point you're just charging a commission rate on top.
**A** (6:43):
Can you tell us a little bit more about this whole delegation process? And like, what, what exactly this means? Is it like, are you talking about DPOs, like what we saw in BitShares on EOS, or is it something different?
**E** (6:55):
Right, so delegation, it's essentially just that if you have in proof of work you want to join a mining pool and you sort of point your mining power at someone else's mining pool, it's sort of similar. It's not exactly like this. As a token holder, you may not have the operational capacity to run a validator yourself, and instead of running one yourself, you have the ability to delegate your tokens, to delegate your consensus voting power to some other validator, and that person then charges you a fee of providing the service, of running the infrastructure for the underlying validator.
**A** (7:33):
So would you say that like the validator operations you guys are running are more similar to mining pools then, like, kind of like you're kind of Acting sort of as a pool operator, but instead of people like sending you their hash power, they're sending you their stake.
**C** (7:47):
They're similar. I mean, the unique things about delegation are your keys stay with you. And so you can assign somebody to be a validator without actually putting your keys in somebody else's hands. You are trusting them that they have to do a good job creating and signing new blocks, but you don't actually have to hand over your keys. And so in the mining analog, when you hand over your hash power, you hope that pool is going to deliver. Here you can kind of see how people are performing and you're not putting your private keys into somebody else's hands.
**A** (8:20):
But in a way, like in proof of stake. So when I give my stake to someone else, right, if they mess up, they can end up usually losing my money that I gave them. But in proof of work, if I hash with a certain mining pool and they propose invalid blocks, at least my asics don't catch on fire or something. Right.
**E** (8:43):
So this is also a distinct. So when we talk about proof of stake, there are many different variants of proof of stake, and they're called NPOs, DPOs, BPOs, DPOs, whatever. So this very much depends on which proof of stake algorithm you're actually talking about. In the tether's case, this is sort of true, where you delegate to someone, and even if I get slash or someone delegates to me, even if I get slashed, that person loses nothing. In the cosmos case, if someone delegates to me and I get slashed, that person gets slashed proportionally. So it all depends on the network we're talking about.
**A** (9:15):
I see. Cool.
**C** (9:18):
So, but I do think, I mean, the point is, choosing a validator is an incredibly important decision because your stake, even though you're not giving them the keys, your stake is at risk. And so you have to know they're going to be up 100% of the time and producing blocks, and they're actually taking the right security measures to make sure that their keys are always safe.
**B** (9:40):
Yeah, I think the operational piece of running a validator is like the distinguishing piece that this is like. This is a complicated operational setup that, like, people who have been in this, who are kind of at the beginning of this space, are investing a lot of time and energy writing software, building DevOps infrastructure building skills, figuring out how these networks get upgraded, how to deal with the various challenges that are specific to a network.
**A** (10:08):
So if I was a delegator and I have a bunch of atoms or tesis or something, how should I like be? And I don't want to run my own validator operation. What are some of the things I should be looking for when I'm trying to choose a validator? So you mentioned uptime security. What are some other nice things I should be in my holistic rubric or something?
**E** (10:35):
So for Cosmos most likely and to some extent also in the future. Polkadot, you probably want to optimize on security because your funds are at stake in those networks. So even if you're a delegator, you can get slashed in something like Tezos. You probably want to optimize for availability because there your funds aren't at stake. So you mostly care about your validator not going down. And then also peculiarity of Tezos is sort of payments don't happen in protocol so you have to trust your validator to pay you at a reasonable rate. So that's like another thing. So it again it depends on the network but I think the major two are availability and security, depending on whether your funds are at stake or not.
**D** (11:17):
As a delegator, one thing I'll add is economies of scale. Some of these networks, the more liquidity and tokens are consolidated to one validator, the more likelihood they have of being successful on the network. So that's something you have to factor in personally as a delegator.
**C** (11:34):
And then I think investors have to deal with things. This is tax. This is when you create new blocks, you get income, you actually got to pay tax on that. So figuring out who's delivering reporting to actually allow you to pay your taxes and understand how much you're earning on the various things or this is kind of all the picture of what's emerging in the category.
**E** (11:56):
Oh actually also one last thing, there's also a bunch of people have started setting up these validators where you have to register with them. I would personally recommend not to use one of those. I would personally recommend permissionless validators.
**A** (12:11):
So sort of like one of the things maybe you should be looking into when you're choosing a validator like validated as like an alignment of philosophy almost.
**E** (12:18):
Once it comes to governance, suddenly cool.
**B** (12:23):
Well I think that's like a good thing. Did you want to segue into the governance question?
**A** (12:27):
I had one more question before that basically I wanted to ask. So you know we talked about like measure like choosing based off security. Right. But like you know, I feel like even the average staker coin holder really like how if I don't have a background in security or DevOps or anything, how do I even judge the security of a validator. How can I even tell who's a good or bad validator?
**B** (12:54):
So we are seeing the beginning of third party auditing coming up in the ecosystem. It's certainly not going to show up before the end of the year. I think sort of one or two services in the Cosmos ecosystem that have like stepped up and started providing security recommendations. We're definitely planning from a tendermint side to communicate more about kind of how we think about security. Validator. We have like a whole validator security maturity model that we're, that we've been starting to communicate about.
**D** (13:28):
This is a little bit subjective, but at Aeon we'll be making direct token investments. So you know, we'll be running that ourselves. And so that signals to the crowd like, hey, we have skin in the game. We're not going to just like, you know, get you slashed and run away.
**E** (13:42):
Probably for now, skin in the game. Besides that, like figure out who these people are and talk to them and ask them questions and then over the long run, Right. This will eventually show itself who runs running secure operations and available operations because those that aren't will either go offline or get stashed.
**C** (14:01):
Yeah. And the last thing I think is there are people who are doing this work and diligencing validators out there from all the big holders and funds are doing the diligence. So in a lot of cases this is going to be a trust business and you're going to have to figure out if people who have hundreds of millions of assets are trusting a set of validators. It's probably a reasonable bet versus two dudes in a server.
**A** (14:24):
Would it be fair to say that it almost ends up turning into a combination of proof of stake and proof of reputation? Almost.
**C** (14:32):
I think it's going to be a huge trust issue. Yeah.
**E** (14:36):
Also this trust will persist across networks, I think. Yeah. Which incidentally may secure networks that have nothing at stake. Because people that have something at stake on other networks are running those nothing at stake networks. But you sort of get this reputational overlap between the networks.
**A** (14:53):
Okay. So I guess segueing into that then like how do you as a validator choose what networks you want to validate? Like you know, do you focus on like one network, put all of your assets on that one blockchain and like, so you can have the highest possible self bond there. Or do you try to like split up your assets across many. Run many nodes? Obviously you probably can't run every single blockchain that exists. So how do you pick and choose what to run?
**D** (15:25):
So, okay, yeah, I'll jump in there. We look at three things. One is the quality of the project, obviously. Is it like a compelling use case? Is there a strong community? Is it a strong code base? The second is the economics. So is it one profitable for us to do? And second, one thing that we've been thinking a lot as we model out the financial projections for some of these things, is the market cap strong enough to support healthy inflation. So there's going to be a lot of price shocks in the ecosystem as time moves on. And that can shake off delegators, that can make validators go offline. It could make the project less interesting. So that's something we're factoring in as well. And lastly is decentralization because we don't want to get caught up in a whole oligopoly thing where we have to collude way too much.
**C** (16:12):
So we work with funds. And so we basically, if a fund tells us they need us to support something, we figure out how to support it. I think validators are going to have to support everything because I think people are going to want to, you know, you're going to want to make one staking diligence decision and find somebody you trust and so that the person ideally supports everything.
**E** (16:34):
And it's possible for validators to run many chains, right? Being a validator doesn't necessarily mean that you want that you expose yourself to the underlying assets. So it doesn't even necessarily mean that you buy large quantities of the underlying assets that you're securing there.
**A** (16:52):
So correct me if I'm mistaken, but I believe Cryptium Labs is the only one who's currently running a live staker on Tezos network. Oh, you guys are too. Okay, cool. And so how did the. Okay, so I guess the two of you that decided to run a live staker on Tezos versus the two of you that decided who decided not to or decided not to yet. How did you make that decision?
**E** (17:15):
For us it was very simple. Our infrastructure was ready. We felt very confident. We ran on the betonet in Tezos for a while. We looked into the economics of Tezos and it makes sense. And at the same time the use case is there and there is a, there is demand for it. Most of all, like there's demand for someone to secure the Tezos blockchain and it's like not fundamentally against our philosophy as Cryptium Labs.
**B** (17:45):
I think like a phenomenally large amount of Tezos is staked right now.
**E** (17:49):
It's roughly 50%.
**B** (17:51):
Yeah.
**E** (17:51):
Which is very decent considering other networks.
**B** (17:58):
I guess my reasoning for not is like mostly occlusion is focused on Cosmos right now and like we're focused on like the Cosmos network launch and so you know, you only have so many hours in the day, basically what it comes down to. And like the Tezos stack is just another stack to get familiar with to build like secure build processes for, to figure out HSMs for and like so we'll eventually get to it because ultimately this infrastructure is very fungible. But it does take like man hours to get a good setup going.
**D** (18:32):
Yeah, first it's just a timing thing. We're building out our platform horizontally and we just want to have our MVP ready for the entire, for all the projects we're going to support rather than just like you know, being specific on every single project.
**A** (18:45):
So I guess going to a point that Zaki was trying to bring up earlier around the concept of governance. Right. So what kind of role do you think validators will play in governance and maybe how will that role shift over time? Because I feel like in the bitcoin community we saw the role of miners and governance change rapidly over time. So yeah, how do you see the role of validators in governance in the short term, medium term, maybe long term?
**D** (19:12):
For me this is one of the things that gets me most excited for a long time. Crypto, even today in a lot of ways is still very passive. For the Dow, the quorum, like it was like 35%. We couldn't even get like 5% on any votes.
**B** (19:25):
Right.
**D** (19:26):
And you know, for Cosmos for instance, I really love the ability for passive delegators to have their vote be represented by your validators or sort of like your constituents. Right. And that is what kind of gets me stoked because you know, it reminds me a lot of like proxy firms for equities. You know, there's big holders out there that don't necessarily want to get involved with the day to day governance. And you know, we can become you know, essentially like on chain like institutions for that that people can look towards.
**C** (19:52):
There's another model too where we will just run white label technology. We're the plumbing and you know a fund who wants to take an active role in governance can then actually take that active role. They may solicit our opinion on technology specific stuff, but they've got a very specific role in the economics or specific governance issues.
**E** (20:13):
So I think in the short term validators will be very, very involved with the chains that I would hope that most Validators will be very involved with the chains of validating on it's like very active in the community trying to understand what the promises are, where you want to take this technology and then I think also possibly actually contributing code contributing let's say key management solutions or like eclipse protection, eclipse protection tooling. So other sort of tooling around the ecosystems they're validating in the long run. My fear will be that due to voter apathy, validators will essentially become these big things that have an undue influence in the governance process simply because I think delegation will be very sticky and I think most delegators won't necessarily overwrite their validators votes.
**B** (21:08):
So one of the things I think a lot about regarding governance is like the validators will have to be very especially in terms of software upgrades. Validators will tend to be very involved. Like you know Tezos put some software upgrade capability in the chain itself. But like as a practical matter validators have had to update in a timely manner in order to upgrade the network.
**E** (21:33):
A timely manner means like someone pinged a message on Slack like in a random baking Slack was like guys, there's a new protocol upgrade, everyone should upgrade in 24 hours. You will drop out of consensus otherwise.
**B** (21:48):
And so and you know, and in sort of safety favoring protocols like tendermint validators that don't upgrade eventually become a veto on the network upgrading. So I think that just like from a practical operational point of view, especially when these networks are young, the validators will have a lot of influence and I'm really interested in how governance decisions that affect validator compensation will play out as a result. It's going to be like a fun thing.
**A** (22:18):
Do you hope, do you think or do you hope to see that like the role of validators in governance will diminish over time like that of miners did or do you think that in the long term they'll continue to have this like large presence in the, in the governance realm.
**E** (22:34):
So I think validators should maintain like validators are one of the players in the ecosystem so they should always maintain some sort of role in the ecosystem. My fears, my hope is just that they don't form oligopolies and like control these end up controlling these networks. I think that's really the biggest worry, like I'm not too worried about valor is not having any role in, in governance. I'm just worried about them having too large of a role in governance.
**A** (23:02):
And how do you like keep up with all of the debates and everything especially as you scale up to more and more chains that you're validating.
**B** (23:14):
I'm worried about this. It sounds exhausting. But I also, I think it's going to be. I think one of the things that's going to be interesting is so like the current generation of protocols is kind of like optimizing for kind of around 100 validator slots. But like Eth 2.0 and Dfinity are all in the 10,000 ish validator slots. But if there's only like a couple of hundred competent validators in the world, well then they'll have to fill those slots and it doesn't really result in that material, that much material decentralization. So I'm really interested in like how will this end up working out? Will we end up in a world where like every computer security professional in the world like runs a validator 2 on the side and that might actually be a very decentralized world and then the role of validators and governance will fall. If like validation tends to be like a very specialized business where like we only grow this by like an order of magnitude in the next 10 years, then validators role in governance will be large because the scarcity of people to run your software that's different. Will be, will be limited.
**A** (24:17):
So what kind of. So you know, we have like a couple of different views of like how the validator space could adopt. There's one which you just mentioned where it's like every security professional is running one and there's like hundreds of thousands of validators or you can have it that it's like, you know, mostly startups that are like doing this. Like you know, most, I think all of us here are like startups or like you know, a branch of a smaller fund. Or do you see like there's a third option where maybe like large established companies in the space start entering like imagine like a coinbase or a binance starts running validators. How do you see that affecting the validator landscape?
**E** (24:59):
Oh yeah. So I think exchanges are in a prime position to start running validators. It's exactly the same problem set of secure and available keys, right? Like when you think about what an exchange needs to be able to do is withdraw funds when user want them, it's essentially the same thing as validators. So I think exchanges will play a large part, will also run large validators mostly because it also offers some interesting economic opportunities around whether they can trade derivatives of the underlying staked asset on their exchanges. So I think exchanges will play rather large part.
**C** (25:36):
I mean think of two exchanges one pays you 5 to 50% annually on your holdings that you have with them. And the one down the line doesn't ultimately the one that's not paying staking rewards is going out of business. So I do think this ends up with exchanges custody providers. My belief is there's going to be a few very large validators that run at scale similar to a Bitmain style approach in the mining world.
**D** (26:06):
One of the things I think about to make the case for startups is the community. So I feel like a lot of people go to exchanges because it has liquidity and security and the economies of scale. But for this stuff there's a lot of trust that's involved. And so for us that's one of the key things we're thinking about is our constituents, our delegators, making sure that they feel like that they want to delegate to us for a particular reason.
**A** (26:32):
What happens when exchanges start offering trading on staked atoms or sorry, stake token, sorry, I think in atoms. But like, you know, how does this, I could maybe trade my like staked atom for your staked tezzy or something. And how does this even change maybe the entire security model of proof of stake? Does it have any effect there?
**C** (26:59):
Yeah, I mean scaled players are going to get even bigger because if you've got, you know, staked tezzies and you got staked atoms, you can swap them for somebody and effectively get rid of your own bonding period.
**E** (27:11):
So I think as long as those assets never leave the exchange, so as long as you don't actually tokenize the state. So if we create, let's say in the case of atoms, if we create like an ERC20 token of a staked atom, I think that's very scary because the economic analysis of how proof of stake gets secured becomes very complicated as long as it stays within the same exchange. And like on Coinbase, I can trade my staked tezzies against my staked atoms, that's fine. Like in the end I'm still trusting Coinbase. It's like my, my security model doesn't change.
**B** (27:43):
So I mean Eth 2.0 and like the one protocol people are really excited about, like protocols that allow for staking derivatives on chain and Eth 2.0 will definitely is like the current spec allows for it. So like we will see this thing that Adrian is so afraid of relatively soon.
**E** (28:01):
It's going to be interesting, like you can start shorting the staked assets and start attacking the specific validator who you shorted.
**B** (28:09):
Yeah, because that creates a world where there's like, occlusion atoms and cryptium atoms.
**E** (28:15):
I think the dangerous thing, though, will be once those two assets traded full par and like, are the same assets to most people, because then people start ignoring the underlying risk. And that's essentially how we got the financial crisis.
**D** (28:29):
Yeah, that's what I was going to say. It reminds me a lot of, like, derivatives, and it's good for expansion but bad for, you know, a beautiful unwinding, as they call it. Right, cool.
**A** (28:39):
Okay, so let's move on to a little bit about, like, the relationships between the people on the stage. What is the, like, relationship between validators? Do you see this as an inherently, like, competitive process or collaborative or something.
**D** (28:54):
In the middle Frenemies? So, you know, we're all here, you know, we want to make sure that the ecosystem thrives. And so we have to, like, you know, be diligent on a lot of things and we have to collaborate. You know, there's going to be a lot of issues where we need to, like, bounce things off each other and, you know, build those relationships, whether that's on the technical side or governance or whatever. But yeah, it's going to be competitive. There's going to be, you know, people are going to be like, you know, competing for fees and it could potentially be a really big space.
**E** (29:22):
I think there are two potential futures here. So one future in which the validators are largely exposed to the underlying asset and are really only validating a single chain. In that case, validators will be very cooperative because no one wants to shoot the guy next to you. If shooting him means I'm risking the reputation of the chain, I'm validating. If validators aren't that much exposed to the underlying asset, I think it will be, like you said, more frenemies where you can collaborate on certain things. Like, how is this, like, what is the tax implication of a val there, for example? But like, you compete for delegation on the same scale.
**B** (30:00):
So I think sort of, at least for the next couple of years, it's going to be very friendly because the distribution of all of these staking tokens is highly concentrated among both sort of people who are insiders or closely affiliated with projects and like, often common across multiple projects. Like people who own Tezos polkadot atoms are kind of highly overlapped and it's going to take a long time for that to sort of unwind. One of the reasons why Cosmos is running this project called Game of Stakes is to simulate what a fully adversarial environment is going to look like at least for a short period of time before we launch the network. And so I'm very curious what that world looks like, but I don't think we'll see it on a main net in the near term.
**A** (30:46):
So if we're a little bit too friendly in the beginning, how do we make sure we're preventing any sort of collusion or anything like that? Do we have crypto economic mechanisms in place for this? Is it a reputational thing? What safeguards do we have against this?
**D** (31:01):
One thing I've been thinking about is just like in general, in an open source community, there's a lot of scrutiny, right? So I think if there's some sort of standardization that's implied, in which all the validators have to be radically transparent, people that are really passionate in the crowd can do their diligence and report on any malpractice which we see on some networks right now.
**B** (31:22):
So we built forking tools into Cosmos to make it possible to fork the network relatively easily. And forking is our last line of defense against collusion.
**A** (31:36):
Cool. And then from the competitive side a little bit. So one of my favorite arguments against proof of stake is this one made by Paul Storsk. He basically makes this claim that in, you know, as an operation you should basically you have capital and you should spend it until your marginal cost equals your marginal revenue. And he basically makes the claim that in proof of work, this takes the form of miners spending more money on hash power and mining more, which is an inherently constructive process where everyone is like, adding to the security of the system. While he says that in proof of stake, validators are going to spend that excess capital on like sort of attacking other validators, like even like hacking each other, ddosing each other. So I'm asking you, I guess I'll ask you guys, how are you planning on spending that extra capital?
**E** (32:34):
So I think the argument, like his argument is very economic, like very theoretical economics. I don't think that's how it will play out in the real world because I don't think that's how it plays out in the proof of work world right now. Secondly, even if all validators spend all their access capital on attacking other validators, they add to the security of the network because it means the resilient validators survive.
**B** (32:59):
I also think what I'm hoping to see in the validator community is validators having significant amounts of excess capital and using that constructively to invest more in protocol engineering and increasing the value of their validator operations by making these protocols more Useful, which miners do not do. Really?
**E** (33:18):
This was something that we are now looking into. It's like. Right, like Val, what I was saying earlier, validators will end up building tooling around the ecosystem. Like releasing for example for Tezos that are currently not a lot of open source multi sig contracts. So for what we also doing is we're writing smart contracts or tezos and open sourcing them to the ecosystem. So I guess that's also how we allocate capital here.
**A** (33:43):
Okay, that's good to know that you guys aren't planning on attacking each other.
**C** (33:46):
I would love to have all this excess capital.
**A** (33:51):
But. So if you guys aren't planning on attacking each other, but you know there are still malicious entities, somebody out there.
**C** (33:56):
Is going to do it. I mean it's worth talking about. The single biggest motivation to attack a validator is another validator to increase. Decrease their reputation and increase yours.
**A** (34:07):
Right.
**C** (34:07):
So this is going to happen and like these guys are friendly and I doubt we're going to be attacking each other on stage. There are going to be attacks on validators.
**A** (34:17):
So how do you protect against these?
**C** (34:20):
Yeah, we, I mean this is what we worry about all day long. It's about, you know, how do you stay highly available. We've got front end DDoS protection that's just for kind of high volume stuff. We've got protocol specific middleware against attacks and we've got a set of, you know, basically hidden nodes that we run so that if our public infrastructure goes down, we can keep up with the chain. And you know, every day it's kind of like how do you brainstorm about a different way an attack might come? Well, maybe look at one that hits us or hits somebody else and adjust to it actually.
**E** (34:56):
So Tesla saw this attack. So someone figured out that you could send, I think, zero. Yeah. Empty messages, empty transactions to validators and it would crash the local validator. And so a bunch of people went down. So these attacks do happen. They didn't come from another validator. Most like they came from a developer that like looked at the source code.
**A** (35:19):
That's a good point. Do you guys as validators like audit the code bases that you're running? Like, you know, what if you didn't do anything wrong, but the code itself had a bug in it that causes you to double sign and get completely slashed. So is this something that you guys are worried about and thinking about?
**C** (35:37):
Yeah. So when we build images, we actually run them through a scanner for some malicious activity. It's not going to catch everything but there's a pipeline for catching some of that and then it's getting mature.
**B** (35:52):
Yeah, I think there's going to be an entire sort of ecosystem that comes around also like the validator middleware, some of it will be open source, some of it will be proprietary to validators where they sort of, where you build tooling that sort of, you know, understands enough of the protocol to make you to ensure that like your keys aren't misused for double signing, et cetera.
**A** (36:14):
And so do you think that like these security setups. So like you know we talked, you mentioned about the like DDoS protection with like hidden nodes in the cosmos ecosystem. We have like we call these sentry nodes. Do you think these like validator security models are going to like kind of converge to like one standard or one or very few standards? Or do you think there's going to be a large like stratification of like security techniques?
**E** (36:41):
I think it will depend on sort of how much the networks converge on their protocols plus the implementations. So if you all converge on LibP2P and we're all running that, that means that we can have a very uniform approach to protect us on the networking side of things because it's all running the same network stack. I personally hope that's not the case. I hope we remain a very diverse ecosystem. Every chain has sort of its own implementation because it makes the entire proof of stake ecosystem much more resilient.
**A** (37:14):
Cool. And so how, and when it comes to. So we talked about the liveness side, what about the safety side? How do you guys make sure your keys are secure? Like are you looking into specialized hardware? Are you looking into like how exactly are you doing this?
**B** (37:32):
So the global HSM situation, hardware security module situation is not good. Essentially the entire industry has consolidated into a single vendor at this point. Gemalto and this is suboptimal for running a byzantine fault tolerant network. So we're really excited about, I'm really excited about like all of the projects in this space that are like oh, we're going to build like a new RISC V based HSM trusted execution environment. But right now like I'd say like there's ledgers products, there's the UB HSM and there's people who are opting for like Gemalta based solutions and there's not a whole hell of a lot.
**E** (38:14):
Yeah. Use HSMs. Remember to back up your keys. Pretty much you have cryptographic keys that if you lose them cost you millions of dollars. You better make sure that you understand against what sort of attacks that's secure.
**B** (38:27):
At very, very least, do not have your keys on the same machine as your validator software is running, because that just makes life gives you like, no defense in depth.
**E** (38:39):
Also, don't use cloud HSMs like you're trusting Amazon with your keys. This doesn't seem to be like a great security model. I think, actually, I think fundamentally most validators should run their own servers in their own data center where they have total control of the hardware they run.
**B** (38:56):
I'd say this is like one of the biggest debates in the validation community is like, how much cloud. Like, there's arguments for lots of cloud, there's arguments for very little cloud, and no one really knows.
**A** (39:09):
Mm. So I guess, I think maybe one last question. So, you know, I guess you could say validators are like, it seems like they're half security company, but also somewhat half like a marketing company in a way where, like we said, we're kind of competing to earn delegation. And so what are some of the different ways that like, you know, you've been reaching out to delegators or like, how do delegators find more? I know some, I know Cryptium has been holding, like, has held meetups, like live meetups. I know some validators create public tooling, like block explorers to kind of get their name out there. What are some of the things that you guys are trying.
**E** (39:50):
From my perspective, do something useful to the token holder. So a lot of the token holders don't necessarily have time to dig deep into the source code of how the proof of stake protocols, for example, work in that specific network. So digest this for them and tell them about how this works, like perform a useful function for them, not necessarily related to delegation, but necessarily related to the asset they're holding for us.
**C** (40:17):
Our core customer is institutional holders of crypto. And so we focus everything around what does a large fund, an OTC desk or an exchange need to run a staking operation. And that's, that's kind of, we'll talk to anybody, but that's our core kind.
**D** (40:33):
Of sweet spot for us since we have like sort of a platform on ramp, we'll have a portal. So it's actually kind of nice in that sense in which we're basically going to be using like generalized, like growth hacking and user acquisition strategies on the web. And that's one thing I'm excited about for validation in that respect.
**B** (40:52):
Yeah, I think there's going to be an interesting sort of space that is like between, like what Staked is doing, which Is like very high touch, like sort of validation service and like the, like. Oh, I'll just go to the crowd. You know it. You know, I think Tezos has like one of probably the broadest distribution of a staking token right now. So it's like appealing to like that kind of an audience. I think some of the other staking tokens are more narrowly distributed and are going to start out with a tighter sort of valuing, high touch businesses. But we'll all probably have to adapt to both.
**A** (41:24):
Cool. I think that's all the time we have, but I don't see Anthony yet, so. Oh, there he is. Are we out of time?
**F** (41:30):
Yeah, we're out of time.
**A** (41:33):
All right. Well, I don't know. I was just going to say, like. No. Does anyone in the audience have any questions that they want to like ask, like about validators? Yeah.
**F** (41:41):
It seems like communication is a big challenge. Not only, like just as Edward said, between protocols and the delegation services, but delegation services with the delegates. Like, how do you communicate new updates, changes.
**D** (42:03):
Yeah. For us, again, since we're developing a platform, we'll have a diligence portal. So essentially our delegates can go up there and we'll provide them with like, you know, governance updates, network health statistics. So we are planning for. To streamline that process for them.
**C** (42:17):
Yeah. So we do something very similar. We've got somebody who combs Telegram and Slack and Discord and all these things and tries to figure out, you know, when updates happen, what are updates to the protocol. And we surface that in our reporting protocol.
**B** (42:32):
Yeah.
**F** (42:33):
If you're a major staker of a chain that comes under regulatory scrutiny, let's say, for offering something that I think considered a security.
**C** (42:50):
God, I hope not.
**E** (42:52):
I'm not based in Switzerland. We'll mostly be fine.
**D** (42:55):
Yeah, We've actually looked into it quite a bit. So we talked to a firm called DXL Law, where they actually pioneered the 2014 decision on miners being not money transmitters. And it's still unclear as to whether validators will be, because the way they likened it is that like miners are randomly hashing and it's kind of like mining for gold. You just dig in the ground. Right. With validators there is sort of that concept, but at the same time, like, you know, you are like, there is some certainties if you have enough capital and minimum liquidity for some of these protocols. Right. So tbd, I mean, you know, the government has a lot on their hands right now, so just operating. We're operating in good faith and doing our best and see as things develop.
**E** (43:36):
Oh yeah, actually work with local regulators wherever you are. That's like they, most of them don't understand proof of stake networks yet and they're very interested in hearing more about it. So like talk to your local regulators and ideally pick a geographic location which isn't necessarily correlated to major economic powers. Too much.
**A** (44:00):
Yeah.
**F** (44:03):
How do you guys choose the network that you want to support? Because pretty much every network that's appearing now as an Ethereum killer is. So what is your criteria with network sustainability?
**E** (44:14):
Maybe actually like, is it open source first of all? Is it something new or is it literally a copy of the Ethereum code base with different license files and slap proof of stake on top of it? So looking at is it a reasonable project really? That's sort of our goal post. Is it something. Yeah. We also wouldn't run closed source software that's like, yeah, run this and hopefully this isn't malicious and hopefully this will do the right thing.
**C** (44:44):
I will run anything my client tells me to run.
**A** (44:51):
All right, well, thank you guys so much for listening to us.
**E** (44:54):
Yeah, thank you.