On-Demand: How to Recover from an IT Disaster
December 02, 2021
By Rahul Mahna
For the second episode of the CAPstone 2.0 Series, EisnerAmper and Datto discussed what to do when an IT disaster strikes and what steps to take to be up and running fast.
Now, we're here today to talk about, as Rahul had mentioned, the cyberthreats and ransomware and attacks that are happening on businesses. Now we all hear about the major stories that happened. Earlier this year, we heard about the Colonial Pipeline attack and that was major news for about a week where a ransomware attack shut down the pipeline. And it was mainstream news. These are the attacks that we can't seem to get away from when it comes to picking up a newspaper or reading an article online and even some other attacks that have happened early into the pandemic.
These bad guys, they have no morals. They were actually locking down hospitals because they were overrun with patients and they strike when the iron is hot for them and they let no stone go on turn. So we saw that they took advantage of cases like that. What we don't hear about on the news and we have to search for are the smaller incidences of this happening. There's cases of providers that are getting attacked and this one that I have for you it's particularly interesting to me.
This case was one of the largest ransomware asked that I have heard of from a small organization. They asked for 1.75 million due to the nature of the data that they had ransomed. And they knew exactly what they were doing when they did attack. They did their homework. So these guys are getting very good at what they do and preparing for their next way of getting that ransom and getting their ill-gotten gains. I know you see this all the time, so what are some of the things you're seeing out there amongst businesses?
Rahul Mahna: You mentioned it really succinctly, Eric. We're seeing that the bad guys are not only using ransomware to lock up the computers and then you have to pay to unlock it, but what we're seeing is they're extracting the data and holding your data ransom and now charging you to not push that data to the dark web. So two facets we're seeing now and it's pretty scary.
Eric Torres: And the fact is it's we're under attack and it's all of us. It does not matter what organization, what vertical you're in. It's attacking healthcare and government municipalities, education, even technology providers as we hold the keys to a lot of the tech advancements that are out there for preventing them from attacking in the first place and then as well as professional services and even financial institutions. The fact is that every single business is vulnerable. These bad guys do not care what business you are in. They will take advantage any chance that they can get, which leads us to our first polling question.
Lexi D'Esposito: Polling question one. Why are businesses under attack? A, it is fun for hackers, b, there is valuable data to extract and resell, c, the employees are gullible, or d, it is easy. And please remember, in order to qualify for your CPE certificate, you will need to remain logged in for at least 50 minutes and respond to three out of the four polling questions.
Rahul Mahna: So as we let people respond to this, I also want to mention that please, if you have questions, we're going to be going through a lot of material here, please enter your questions in the chat section. We'll do our best to get back to you during this. If not, we will absolutely get back to you afterwards, and someone part of our team will follow up. And for those of you that might have to cut it short, I really want to encourage you to stay to the end because Dan is really going to do something very interesting and simulate an actual disaster and then show how you can have a business continuity plan and show how you can be up and running within minutes if you have thought through a system properly using tools such as Datto. So please definitely stay to the end. You won't want to miss this.
Lexi D'Esposito: All right and we are now closing the poll and sharing the results.
Rahul Mahna: Fantastic. Eric, keep educating us.
Eric Torres: All right, so the next step we have here is, if I can advance, so why are cybercriminals targeting your organizations? And the fact is it's the amount of data, the useful cache of data that they do have, a sensitive data. Some of you may have healthcare information on there, if you're processing credit cards. All of these vulnerabilities and all of this data that is valuable to these cybercriminals because they can go out there and sell that on the dark web. And then let's face it, a lot of the smaller organizations don't have the budget or the proper planning that some of the large enterprise-sized companies have when it comes to protecting their networks.
Now we get into, "How often is this happening? How often is a business hit with ransomware?" And that data, we have a ton of stats. We survey our own partners. We survey even small businesses, small-to-medium-size businesses that are out there and the stats that we are finding are quite frightening. And you can read through the stats on this page, but if you go right down to the bottom one, a business is hit with ransomware every 13 seconds. That is how prevalent it is. That's how often that this is happening. It's happening so often that actually our own government is in talks right now to make it illegal to pay ransom. And they look at this saying, and it's the Department of Treasury about a year ago, a little over a year ago today, they came out and said, "We're actually thinking about making this a federal crime, much like it is in a ransom case where a kidnapping case it comes into play. It's the same exact sense. They are holding data for ransom. And if you do pay, essentially you are funding terrorist organizations."
Rahul Mahna: Eric, I'll just jump in for a second that as part of our EisnerAmper Digital team, on average, we're getting at least one to two requests coming in that they've received ransomware. They've been hacked. They've had some kind of nefarious activity and that's been picking up ever since COVID. It keeps increasing.
Eric Torres: Now in order to understand how all this starts, we first have to understand the anatomy of the internet and what does it really look like. Where you and I live on a daily basis, where we do our shopping, where we get our news from, where we waste time, that is the surface web. And just think about the vastness of that. That is where we do most of our things. Anything behind a security layer, that is the deep web. That's your financial records, your healthcare records, academic records, anything that's behind that barrier and there's a lot of movement that happens within that and then we get to the dark web which is a deep and scary place. If you add up the deep web and the dark web, that's 96% of all web traffic. So think of that, how we think of how large the Facebooks and the Googles and Yahoos are of the world, it's that much bigger below that and that's where these bad guys lurk.
Now, let's talk about the evolution of these threats. Who are these threat groups and really how are they getting in? So we know that ransomware developers, they are making a ton of money doing this, billions of dollars every single year. And what we also saw was a shift early on in the pandemic as businesses started tightening their belts and laying people off, furloughing people, cutting back hours, we actually saw an uptick of insider threats, those that were opening a door up intentionally, or in some cases, Datto actually caught and then we reported them to the FBI. There was an employee that went on the dark web and attempted to sell their credentials to their network. And these are real threats that we have to worry about.
Also, since there is this amount of money changing hands, we know that organized crime has their cut in it, and when I say organized, believe me that it is organized. These are full-blown companies, if you will. The ransomware developers, the ones that are out there accepting the money and hiding the code and spreading it, these are organizations with even storefronts and businesses and offices that they actually go to. These are not lone wolves in their basement that are just sending this out, hoping that they make some money. This is very, very organized. And more often than not, it is coming from different nation states that tend to not like the United States and we are a target for them.
Outside of that the biggest threat that I am seeing right now and then what I think the future of these threats will look like are all of the internet-connected devices that are now introduced to the network, especially since all of us want to go work from home at a moment's notice and many of us are continuing to work from home and just think about all of the devices that are connecting to your network that now introduced a vulnerability. So how are they getting through? How are they attacking and getting past the security measures that are in place? And the fact is that they're doing it in mostly unsophisticated ways because these are tried and true methods of getting through.
The first one is just brute force strength and that's a password cracker. I mentioned the dark web, and now on the dark web, it's about $20. You can buy a password cracker and the compute power behind it used to be immense way back in the day. Now you can crack a password in a matter of minutes, what used to take days. Now once they do crack that password, the first place that they're looking is where your stored passwords are, your browsers. And how many of us, for the sake of time, store our passwords right in our browser? That's the first place these bad guys look because they know that oftentimes your passwords are repeated and you're using that for work or for maybe your banking or maybe it's your purchasing online.
Another way that they're getting through is called credential stuffing. And what that is basically taking existing passwords that are known passwords and selling it on the dark web. If you haven't changed your password in a certain amount of time, in a long time, there is a chance that is living out there on the dark web. And like I said before, a lot of us reuse our passwords. We have one password for the same thing over and over and over. That one password from a site that you may not have used in three years, it may be the same password you're using for your banking as of today.
And then lastly, the number one method that these criminals are using to get into networks is phishing, sending an email out and getting somebody to click on that email. They have advanced phishing attacks where they're even making phone calls and placing phone calls and following up to that email, "Hey, I sent you an email. Will you click on this link?" All of these introduced the vulnerabilities and the ways that these bad guys get access to a network. Now we know that ransomware is king, and quite frankly, these guys are not going to stop. And it's because they're making so much money doing this and it's billions and billions of dollars of just what is reported into the FBI.
And I think this these numbers are incredibly low because we know that there are a lot of organizations that are not even reporting when they are getting attacked. And this leads us to our second polling question. So Lexi, take it away.
Lexi D'Esposito: Polling question two, what is the most frequent way hackers penetrate a business? A, they knock and walk into the office, b, denial of service attacks, c, send a fake email that makes an employee click or, d, steal credentials at Starbucks pretending to look at your coffee. Please remember, in order to qualify for your CPE certificate, you will need to remain logged in for at least 50 minutes and respond to three out of the four polling questions.
Rahul Mahna: So Eric, I wanted to just go back to your comment about the dark web. You've mentioned you've actually seen it. Can you let folks know? They get confused when I say dark web. What is it? Is it a website you log into? How did you get access to see it?
Eric Torres: I saw it once and it was extremely frightening. I was at an event with one of our engineers and I just asked, I said, "Do you know where the dark web is?" and he showed me and there's proper protocols he had in order to access it and make sure that we were safe. And it's basically a giant trading expo center, if you will. It's a trading floor where you can literally find anything you want. Anything you can think of, you just search for it and it pops right up and it's for sale. And that comes with malicious code. You can buy that. You can buy just about anything else. And it is incredibly scary. I was on it for all of 10 minutes and thought that that was enough for me. I've seen it all and it's all I needed.
Rahul Mahna: We offer to our clients that we monitor the dark web for them, checking their email accounts, checking their domains, watching out for them, because like you said, you can by password crackers, Social Security numbers, email addresses, passwords. I probably haven't even mentioned the good stuff that they're going.
Lexi D'Esposito: All right, and we are now closing this poll and sharing the results.
Eric Torres: There we go. Mostly everybody got it right. 95% with the phishing.
Rahul Mahna: All right, not Starbucks.
Rahul Mahna: I thought somebody would pick the Starbucks coffee one.
Eric Torres: All right, so what it really comes down to is your people. People are your greatest risk. If we know that phishing is the number one way that they're getting into it, it comes down to training your people, so that we know what to click on and what not to click on. And we never want to be the one to accidentally open the door to the cyberthreats that are out there. Ultimately, it's our job to help you stay protected. And the first step in doing this is being able to identify the components of an IT disaster, knowing what you have on your network and thinking about all of it. And that data we run through these exercises ourselves, the ones that tend to slip through the cracks the most of the SaaS applications, all the little widgets and applications that people are using that most might not be aware of. So it is key to identify absolutely everything in use and developing that process. Once you know what you have, it comes down to creating that disaster recovery process and knowing the pieces, the components and then what to do if and when there is an attack.
And then after you have that process identified, it then comes down to identifying the people and the operations side of it. So what do your people need to know as far as if they go down, what to click on, what not to click on. And this is where you guys shine as far as helping with the training and helping people to understand what to look for, what to be on the lookout of when it comes to these phishing emails. We all get them. I've got a whole file folder full of ones that come through, but it does come down to training. And the key is this is that security isn't just your IT person's responsibility. It's not your IT department. It's not your outsource department. It's everybody's responsibility. Everybody has to make sure that they're vigilant, staying up with the times and concentrating. Security isn't something you just purchase, it is ongoing and you have to invest in it and you have to make sure that you are staying a step ahead of these bad guys.
And then lastly, it comes down to identifying the technology needed to mitigate these risks which is why we're here today and where Datto shines when it comes to making sure that your networks are alive and viable if and when something bad happens, that if you go down from a natural disaster or from an unplanned attack, the bad guys get in, how do we recover that data at a moment's notice? Aside from that you have wonderful security solutions that you offer and keeping the bad guys out. Look at that insurance policy saying, "If they do get through, our data is protected and we can come back at a moment's notice."
Now when I do talk to people, I hear this a lot, I already have a fantastic backup solution, "My data is already off site. It already lives in the cloud," and that's fantastic. Your data is in the cloud, but that's not necessarily a true business continuity solution which leads us to the third poll question and Lexi, take it away.
Lexi D'Esposito: Ransomware is the core tool hackers use in their efforts. How can a business prevent downtime? Please select all that applied. A, have a disaster recovery plan, b, have a cloud backup of your entire server, c, educate their employees on cyber phishing techniques or, d, Always be patching, updating and running multiple security tools. And please remember, in order to qualify for your CPE certificate, you will need to remain locked in for at least 50 minutes and respond to three out of the four polling questions.
Rahul Mahna: So Eric, you hit on something really interesting that I hear often is, "I already have a backup. I already put in my USB drive. I already have a computer in my office I back up. I put my files here there. I have a $5 a month cloud backup service. I'm putting it on Google Drive. Isn't that safe enough?" But that's not, to your point, a business continuity plan. It's probably not even a well-thought-out plan. So I know you're going to talk more about that. Dan's going to show a little bit more about how do we have a business continuity, keep running your business.
Eric Torres: I feel that anybody can provide you with a backup solution. Literally, we can plug a thumb drive into your USB drive and into a computer and back it up. And we can take that off site. And technically, that's an offsite storage solution, but what it really comes down to is recovering that. What happens if your systems go down? And it all comes down to mitigating the amount of downtime you have. And thinking about if your computer network, if your network just shut down at a moment's notice, how long until you can recover that and bring that data back over? And that's where we start talking about the true business continuity solutions that are out there.
Lexi D'Esposito: And we are now closing the poll and sharing the results.
Eric Torres: All right, so it's right about even. It's all of them.
Rahul Mahna: It's all of them. That's really it.
Eric Torres: So what this really comes down to is, do you have the backup solution or a true business continuity solutions because they are completely different. As I mentioned, anybody can deliver a backup solution. It's taking your data and putting it somewhere else. And due to the ransomware that is running rampant, you have to decide, is your solution just a backup or can you recover at a moment's notice due to that ransomware that is running rampant? And this is where you guys shine. And this is where you guys are truly doing something about it for your customer, and making sure that they have an option to have true business continuity.
The easiest way I can describe business continuity is this. Through the power of our companies, we rewind the hands of time. If something bad happens, if there's a fire, if there's a natural disaster, if there's a ransomware attack, what we can do is roll back the hands of time to five minutes before that attack happened, five minutes before that fire started and get that network back online. Just recover from that specific moment in time. So here's what it looks like and I'm not overly technical and I won't bore you with getting too far into the tech, but this is the system in action.
On your screen on the left hand side is your critical business systems, your servers, your workstations, how you conduct business in a daily manner. What we do is we take that data and we move it over to what we call our SIRIS Solution. That's a brand name for the business continuity device that lives on the network and that that device is backing up the data as little as every five minutes and it's also running through a series of tests, which Dan will show you shortly, where we're running through verification tests, for screenshot application verification. We're even looking for ransomware.
If there's any footprints of ransomware, at that moment, we're able to let you know, let your team of engineers know and raise a red flag saying, "Something's wrong with that last backup." In addition to this, we're able to restore from that on-prem device. We're able to restore that data, virtualize it at a moment's notice, which you will see here shortly and then we're also taking that data and sending it to two of our data centers, our primary and our secondary data center. So we have your data essentially in four locations at all times, your actual data, the data lives on the data continuity device that's on your network and then the two data centers.
And as an added feature, you can recover that data from our data centers at a moment's notice in a matter of seconds if you lose the entire building. So it's a true hybrid approach to how we're backing up and storing that data. This leads us to the fourth polling question. And Lexi?
Lexi D'Esposito: Downtime can dramatically impact an organization. What is the best way to prevent organization downtime? A, have a cloud backup of all your services on a separate network that can be used in a planned rapid manner, b, copy your files to a computer in the office, c, back up to a USB stick because you're a small organization and no one will attack you or, d, give all users administrator access to their computers so they can self-manage risk. And please remember, in order to qualify for your CPE certificate, you will need to remain locked in for at least 50 minutes and respond to three out of the four polling questions.
Rahul Mahna: So I know we're moving along really quickly here. And I appreciate all the download. Again for those folks, please put your questions in the box. I know you probably have a lot of them from past Capstone Series. We always end up with many, many questions. So I want to make sure we address them. If not today, we'll follow up with those as well.
Eric Torres: Sure, excellent. If there is time at the end, yes, we can address some of them, but if not, rest assured, somebody will be in touch and we'll be able to take care of you.
Lexi D'Esposito: And we are now closing the poll and sharing the results.
Eric Torres: All right, 91%, have a cloud backup of all your services on a separate network that can be used in a rapid manner. And this leads us to coming up to our technical demo and showing what this really looks like through the technology and what it looks like in action. So real quick high level of what the solution does, you're working and your end users are working. And for whatever reason, they clicked on a wrong link and ransomware gets introduced through your network. What this solution does is we simply go over to the Datto appliance, the on-prem device that lives here, this one that's right here next to me and we spin up whatever device was infected, your server, your workstation, and we virtualized, we get an exact same replica copy of your device running in a virtualized state, so that you can continue to work.
And the benefits of this, it reduces downtime for everyone. We can recover in a matter of seconds, which you'll see shortly. We can recover from both a local copy or the cloud copy, depending on what is happening with the network and your needs. And then lastly, this helps with, and forgive the industry buzzwords, but it helps with your RTO and RPO. That stands for recovery time objective and recovery point objective. Basically, what that means is the cost of downtime for you, what are your thresholds when it comes to that investment in downtime and this helps mitigate that true cost to you.
So with that, I would like to bring up my good friend and one of our senior solutions engineers, Dan Ciccone, to walk us through what this really looks like in a real world scenario.
Dan Ciccone: Thank you, Eric, and good morning, everyone. My name is Daniel Ciccone. I've been with Datto for just over a nine-year mark. Prior to working here, I have about 12 additional years of IT experience and I want to be very candid with you. I've stayed away from backup almost my entire IT career. There's nothing fun about backup. As I started working for Datto, I began to understand that this is so much more than just a backup solution. And with that, we need to understand how we bring data into the system, how it's stored here, the security and the immutability of that.
So in our little demo setup here, I want you to picture that this system up top is the server. This is the potential Datto system on site on network. And first, we'll just describe how data is brought into the platform. It's an image-based backup. That's very important to note. If you've only backed up files in folders and the server goes down, the time that it would take to reinstall your operating system, all the applications and then put the files back to where they would need to go, that would cause a significant amount of downtime. If we only have a traditional image-based backup solution without the failover component, we're subject to downtime. The more data we have, the longer it would take to recover the system.
The interesting thing about that comment there, if you've only taken a backup, and granted, you could have your backup in another location, say your system gets hit with ransomware, the time that it would take to restore it may be more expensive to your business than the actual ransom. We're not encouraging you to pay the criminal or saying it's a much better idea to have this type of insurance policy where no matter what happens to your server or your infrastructure, we can run that server off of the appliance inhouse or in a greater disaster like my first week at Datto, Superstorm Sandy here on the East Coast. We can virtualize entire networks in our data center and get you remotely connected from anywhere that has power or network.
With that, we're going to dive into the interface of my appliance here. And from Rahul's team's perspective, for any device deployed within their fleet, they have a monitoring platform, the solution tells us so much more than your backup completed. If you're only taking a backup and we only know that the backup finished, that tells us very little about recoverability and the amount of testing that goes into backup verification can be extensive. You're talking about hours and days and weeks' worth of investments and time just to make sure that that restore process goes smoothly.
Traditional backup solutions, legacy backup solutions usually make us go like this and hope that our restores are going to work. There's not much assurance there. As we're monitoring this demo environment, we can see that our backups have completed, that they dropped to the appliance about a half an hour ago, they've replicated to our cloud at the same level. That can happen on a daily basis, but I like showing this off. We have each local backup going into the cloud consistently, and on a daily basis, SIRIS is actually starting from backup a copy of each production machine first in this testing process just saying, "Hey, if you go down, I know I can get a copy of your production machine. If there were an error here, we can take this proactive approach to troubleshooting."
So SIRIS tells us a lot about what's happening prior to needing to failover and we can take a proactive approach to troubleshooting which is a really interesting concept. We're not just starting those production machines here. There's another example of a screenshot that ran this morning. Let's go into a little bit more detail on what SIRIS tells us when every single backup completes. Automated verification, that checks to make sure first that all of the volumes are actually protected. The solution is not protecting files and folders. SIRIS backs up the entirety the operating system, the C drive, all of Windows, any application, every single file and folder.
This is a block level image-based application aware backup. Very technical terms. Image based means we backup everything. Application aware means that we're able and meant to protect database systems throughout the course of the day with hardly any impact in performance, whatsoever. That also goes to the block level concept here. Legacy backup software would need to traverse the entirety of your hard drive looking for changes that have occurred. And with that, there was a significant impact in performance. So you were likely only getting one good backup a day. That's not what this system is designed to do. Very intelligent software designed here at Datto that pays attention to what's changing on the server at a kernel level so that we can run our backups throughout the course of the day.
Eric had mentioned this, we typically recommend hourly, but that drops all the way down to a five-minute interval. When every single backup completes, we check to make sure that each volume is actually present here, that we can see the C drive, the D drive and so on and so forth, that our volume shadow copywriters are healthy, the file system has been correctly verified, and last but not least, there's no ransomware on the image. Now before we talk about detection of ransomware, there's another incredibly important thing that you need to consider a backup. Where do your backups live on your network today?
We've been talking a lot about ransomware. These guys, these hackers are not stupid. This is a multibillion dollar a year industry. And with that, they're not only seeking out to encrypt your servers. That's half the battle. As of late, more and more often, they've been seeking a path to traditional what I call dummy backup storage that's network-attached storage, direct-attached disk. Anywhere within your SAN, if your backups live within the same storage where your virtual infrastructure lives, that's incredibly dangerous. They know if they get to your backups, they're getting paid.
And I have some really interesting news to share with you about SIRIS overall. First, this is a Linux-based system. The majority of ransomware is running for Windows. That's not the end all be all. Every bit of our infrastructure, including all these systems and every bit of our cloud infrastructure, everything Datto is built on a very intelligent file system. It's called ZFS. The zettabyte file system was first developed for incredible amounts of data. It's also incredibly intelligent and that it simply cannot be corrupted by ransomware. Now I know this is an incredibly bold statement to make. If anyone would like to get into the technical reasons why this file system is literally immutable and incorruptible, please schedule some time to talk with us. We'd be more than happy to do so.
There's a little analogy I've been using about ZFS as of late, if we remember the CD-RW technology, those CDs that you can write to over and over again, that's like Windows. You can write and save to Windows. That's what it's meant to do. Our file system is a bit more like CD-R and that it can only be written to initially and then it checks itself consistently. So data integrity is a key characteristic of what we're doing here. So while your systems can become encrypted and I can take a backup and say, "Wow, that last backup looks like it has ransomware," as we go to recover the system from an earlier point in time and I'll show you this on our recovery page, it would literally say, "Ransomware suspected." That's half the battle right there. We do not need to do full system restoration to understand where we're healthy. Therefore, we can move backwards in time pretty seamlessly to understand where we're going to virtualize your server should something terrible happen.
With that said, all of the verification running here on a backup basis, we also have a bit more to talk about. When screenshot is complete, we don't just make sure that the operating system has started correctly. We can verify Microsoft applications, essential services. We can run scripts against these virtual machines as they spin up on a daily basis. In my very humble professional opinion, there's no one that paints quite a clear picture as we do that the restore is going to work, that as we attempt to fail over, as we go to do these operations, we're not going into them blind. We can do them with more confidence, but you talk five, 10 years ago and you're crossing your fingers in every type of recovery and that's a little scary.
We mentioned a bit about Datto Cloud. The system is set to take your local images and replicate them out to our bi-coastal data center. We even have seating in place for that. So we can take the lion share of the data, replicate that offsite. What's very interesting about the underlying technology here as well is we only take what's changing after the fact. A lot of backup solutions, they make you take another full image occasionally. That can mess with your off site replication. You might expect some impacting performance on your network. When this is set up and throttled, it's seamless. Rahul, I'm sure you can speak to this as well for the people that haven't deployed already. Did they notice that SIRIS is running on their network?
Rahul Mahna: No, most of our clients do not notice it. It's a small device. There's even smaller devices that sit in our client's infrastructure that they don't even know exist and it just seamlessly works. And we have hundreds and hundreds of these devices in our clients now.
Dan Ciccone: While we are image based and we do provide failover and that is the main concept of the solution, "If your server goes down, I can start a copy of it here nearly instantaneously," realize that we can do very simple operations as well. Some of the more common things that may happen, someone's deleted something by accident or perhaps something was deleted maliciously. We can either go in and do a straight file restore of that from any backup point. I'll show this off very quickly just to outline some of the speed and benefit of what we're talking about. From a previous backup of this server, that's how long it took for me to make the entire C drive available. This one only has a C drive, but very quick, I can come in here, click and download data. That's speed, ease of use, key benefits of ZFS.
Next, we're literally going to simulate a disaster. Now we don't have a ransomware that we can infect here, but we will say, "Hey, no matter what happens to that internal system could be ransomware. It might be some other software failure of the solution or in a hardware failure solution or downtime caused by bad drives or what have you. Think about having traditional backup. Rahul may need to deploy technician, assess damage, order parts, wait for that to come in and then initialize the recovery process that would lead to more downtime.
In the event that anything happened to your server, whoa, no matter what happens to it, we can simulate what it would look like to get that system up and running on our device. So picture this system in your infrastructure, we can remote into it from anywhere in the world. From any of the images that we've taken, which typically run hourly, we're going to start creating copy of that production machine. And this is the main highlight of the technology that we've developed, the time that it takes to get that system back live on your network is just absolutely incredible.
While there are competitive solutions out there in the world and some claim to have the same ability, we can do this despite the size of your server, despite the number of backups that we've taken or moving backwards in time. So just that quickly, folks about six seconds flat. We have a booting copy of our production machine spinning up on SIRIS. There is some light networking that we need to take care of. We have to reassign an IP address. I want you to picture your original system being dead in the water, SIRIS becomes your production machine, all of the changes working here are being captured. We're now backing up this virtualized system and we can take all of those changes internal, bringing them from serious into a new bare metal system, perhaps you wanted to upgrade your server and the original was the Dell and you want a brand new HP, we can do that, no problem.
We can also simply roll the system backwards in time as well. From ransomware perhaps, we can just say, "You know what? The system had a massive amount of data change and I want to roll back to a previous hour." We have that capability as well. Last but not least, in the event of true disaster, and speaking to my first week at Datto when we were a very small organization, only about 150 employees, the majority of us support and development, we had to literally undergo our largest undertaking at the time. There's about 300 different businesses that needed a virtualized in Datto Cloud. The recovery launchpad is Datto Cloud.
I just want to show a quick example of how long it would take to build a couple of virtual machines including a workstation here in Datto Cloud should true disaster strike your environment. And again the speed at which we can do this is just absolutely incredible. This will speak to the testing and verification processes that we allow our partners like Rahul to go through here. So imagine prior to disaster, and as soon as data is seated in our cloud, in the moments that I've been talking, I've already created three different virtual systems in our data center. And before disaster strikes, we can outline the order to bring up the systems, networking, remote access, where would you potentially be working from.
Having a plan for this is paramount, of course. And with the time that it takes to do this, folks, setting an expectation that this network is available in less than 24 hours is very, very easy from our standards. It's usually about a half an hour to a couple of hours from disaster to the point that we would be able to launch and connect to systems in our data center. It's the same thing happening in Datto Cloud. Awesome.
Rahul Mahna: Can I poke you for a couple questions?
Dan Ciccone: Please absolutely.
Rahul Mahna: So I know you spoke of a lot of technology, a lot of different things, let's just make it really simple for some of the examples that we've been seeing.
Dan Ciccone: Sure.
Rahul Mahna: So recently, we had a client whose environment got hacked. They had ransomware, it locked up all their computers and their servers and they wanted to replace their equipment. Let's talk just the server.
Dan Ciccone: Sure.
Rahul Mahna: Our problem was we couldn't find the server for two weeks. As many folks know, supply chain issues also is impacting our IT industry.
Dan Ciccone: Absolutely.
Rahul Mahna: So they had no server for two weeks. So going back to what you just showed, could they have been up and running within a few minutes?
Dan Ciccone: Absolutely. So the original systems, despite what had happened to them are offline completely at this point, just so they don't infect anything else or what else could have happened to them. We build these solutions with enough resources in the way of power to say, "Okay, what is your largest individual single point of failure internally? What would I potentially need to run from here and these also have the capability of talking and running from the cloud, even for internal disaster?" So beyond what's built here, I can start systems from the appliance, I can build them back to their hypervisors if they're healthy or we can actually spin up in Datto Cloud. There are plenty of resources to go around here.
In that scenario, you're likely talking about a matter of however long it takes to start the system, reassign an IP address and reboot one more time. That's it. That is the downtime they would incur.
Rahul Mahna: Our team does that for all of our clients. We do our planning, as Eric said. We do our risk mitigation. If we thought this through, a server could be back up and running in a matter of, let's say, a few minutes, an hour and even workstations. If we plan correctly, -
Dan Ciccone: Absolutely.
Rahul Mahna: - I just want to be clear, we could do workstations as well.
Dan Ciccone: This system is compatible to back up, everything dating back to server 2003 which is incredible to say a lot because not even Microsoft supports that these days, but yes, I do have a Windows 10 box right here. So potentially, all they would need is a spare workstation to connect to this one if the original had a hardware failure. Should it be something like software, it's just going to get wiped eventually, but this would act as that system temporarily. It doesn't take much to reinstall Windows these days and just connect here, so that's great. We would just wipe the original.
Rahul Mahna: And back to one comment you made just for clarification, ransomware. If it had infected our clients, does that ransomware come to the data ecosystem?
Dan Ciccone: There's literally no way that it can happen. I know that is again a very bold statement to make, but considering what we built under the hood here in terms of security and I should mention that we have the ability to offer full encryption for the entire solution as well, but this system is literally immutable when it comes to ransomware. And let's say by some stretch of the imagination that it were compromised, it's not going to traverse into Datto Cloud at that point. There's an air gap between the solution in the way that data communicates offsite. So we would be well aware of anything happening there.
Rahul Mahna: That's fantastic. And a lot of our clients, we're very sensitive and our team to handle our clients that have five users as well as 500 users.
Dan Ciccone: Sure.
Rahul Mahna: These solutions can work across the board, correct, for small and large-
Dan Ciccone: Worth noting, yeah. So what we're looking at here is about a one terabyte system, but they do range from one terabyte out to 100 terabytes. In some cases, we look at the number of systems that need protection. We have architect along with you guys to say, "Hey, maybe two SIRIS make sense here," but it does scale. It was built for the small-to-medium-size business because if you think about the enterprise space, they usually have what it takes to recover from hardware failure. Ransomware changed the name of the game for everyone. It sure did because while you could have hardware resiliency and systems that can fail over to one another, you need a way of effectively moving backwards in time and that really only comes with continuity.
Rahul Mahna: That's great. We're really excited about these devices and how they help our clients with the continuity problem mostly which is what we're impacting with the hardware shortage right now and getting our clients up and running right away. This has been a game changer for us.
Eric Torres: Awesome. Well great. Thank you, Dan.
Dan Ciccone: Welcome, Eric.
Eric Torres: And for anybody that wants to think of it in a real-world scenario, the disaster that was caused here, the fire that was started, that's your systems going down. And in a matter of, what? We've been up here for 10 minutes with you Dan and you're able to recover that device, not only recover it but then spin up three more saying, "Here's your network. Here's how you can get back up and running."
Dan Ciccone: And that's saying it's SIRIS and everything along with that infrastructure were destroyed, yes. Bring it up in Datto Cloud.
Eric Torres: A matter of minutes of downtime or even seconds of downtime rather than days or weeks depending on supply chain issue. Now in thinking of all of this, we must consider the cost of what downtime is. And there's a ton of stats that are out there and articles that I pay attention to and even some surveys that we do ourselves here at Datto, but the fact is this, just in the past two years alone, the cost of downtime has skyrocketed to 486%. Now, you have to consider all facets of going down, not just the ransomware ask, the lost productivity, supply chain issues, paying somebody to come back in to revamp everything, the time it takes to transmit that data or transfer that data to a new device, all of that starts adding up and the meter is running.
On top of that what a lot of organizations are not thinking of is what do you have to do to make sure that it doesn't happen again. So there's forensic investigation. There's consulting fees. There's a lot that really does go into it and that starts adding up time and time and time again. And what we have is a recovery and downtime cost calculator. And this is a takeaway for everybody tuning in. There's nothing to pay for. It's on our website. This is where you can sit down and in a matter of five minutes, get just a ballpark idea of what even an hour's worth of downtime cost you and it's super easy.
There's only 10 questions and it's questions about your thresholds for how long you can be down, how many hours, how many minutes, how many days and then what are you recovering from. How far backwards in time are you going? Is it an hour ago? Is it yesterday's data? Is it last week's data? And then there's questions about your business itself. And the network that is on within your business. So the amount of data you have. How often you're backing up that data? What does the recovery process look like? If they need to scramble the jets and get engineers out there onsite right away, how long until somebody can be out there?
And then lastly, how many employees are affected by this? Are we talking 20 employees, 100 employees, 500 employees, as you mentioned? And we also take into account the revenue and the stop in the revenue stream coming in, the lost productivity of these employees during that downtime. And then it lets you a ballpark number and it compares true apples to apples comparison, our business continuity solutions versus your current solution that is out there and what that means from a to cost standpoint of being down for every hour that you're down.
Dan Ciccone: It was like saying that these systems are the employee's absolute worst nightmare. The administrators should embrace it wholeheartedly, "You're not going to have downtime." The employees are never going to hear, "Hey, the server's down. Go home with the rest of the team." That doesn't happen anymore.
Eric Torres: So I was wondering where you're going with it.
Dan Ciccone: That's something we hear quite often about serious. Employees hate it.
Eric Torres: All right, I know that we covered a ton of information. We had a great technical demo. We talked about the risks that are out there, mitigating the risk. And what it really comes down to is taking that next step for a needs assessment, a security assessment, an audit of what's happening on the network or the downtime cost calculator and this is where you and your team shine and helping them understand what's on the network, what flaws that are out there, what risks and vulnerabilities that are out there. And some of you tuning in, you may be thinking, "I've got a great provider. I've got a great in house IT team that does this for me." That's even more of a good time to have your team in there, a second set of eyes. Maybe if nothing more than to say, "Yes, you guys are doing great," or, "There's some vulnerabilities that we found."
So for all of this, and I will share your email address, for all of this information email Rahul and any parting shots on any of this that you can share with again?
Rahul Mahna: Yeah, so I think that's a wonderful way to finish the webinar in a sense where you really should begin by measuring your risk. So our team at EisnerAmper Digital, after many, many years of working on this problem, we've created our own risk assessment tool called First Look. And it's a really good drill-down tool for the SMB market where we really hone in on where's the risk right now. As Eric mentioned, as Dan mentioned, security is a multilayer approach. The layers of an onion, I call it. So you need a team of folks that come in and are familiar with, "What are those layers right now? Where are the trends? What are we seeing?" and we've modified our tool to provide that.
And for anybody that's interested, we'll offer that for free for you joining this webinar with us because we're really excited to help you in trying to find these gaps and trying to mitigate them as well. So I think this was a fantastic webinar, in my mind to try to provide our clients a sense of how can you not just have a disaster recovery. I really don't like that term anymore. I'm really moving more towards business continuity. If we've been planning right, we've been doing our assessments, we've been working with you, you should be able to continue your flow of your business.
As Dan said, the employees won't be happy because that server never should go down. The computer at the administrative desk should never go down. The computer at the CEO's office should never go down. Within a few minutes, everything should be back up and running if we've really been doing our part. So I hope that that has been part of what you've taken away from this. I hope there's been some knowledge. We had some fun lighting things on fire. And that was real fire, I could smell it. Trust me, that is not fake.
Eric Torres: I was waiting for something like alarms to go off there.
Rahul Mahna: But I would like to just take, we have a couple extra minutes. Eric, I don't want to put you too much on the spot, but I just wanted you to talk a little bit about risk. We're talking about measuring risk right now. We're talking how to manage and monitor it, what Dan had showed, so many tools that our backend engineers use, so much facilities that are provided to you as clients and we all talked about ransomware. But that is going to change. As we're today talking about ransomware, as we all know in the IT industry, it changes every few months, every few years, the risks change. Eric, just to put you on the spot a little bit.
Eric Torres: Sure.
Rahul Mahna: Could you talk about some of the evolution of that risk that is happening? Is it going to stay ransomware? Where do you think it's going to go and what are you seeing as Datto?
Eric Torres: That is an excellent question and you used the word evolution and I'll share with you just where ransomware has gone in the past couple years and how it evolved and then where we're looking at it going. So ransomware used to just be something that they deployed on a network and it locked up systems and they held it for ransom. And they started getting creative with introducing variables in the ransom itself. For example, depending on where you live, they would ask for a higher ransom ask. So they were able to have location-based ransomware. That was one of the evolutions a couple of years back where we're going, "Holy cow, these guys are really taking advancements in doing this."
There's another evolution that we saw a couple years back where it wasn't a one-time ransom ask. It was saying, "I have your network locked down," and it was a monthly ransom subscription model, if you will. Instead of asking for $30,000 upfront to recover your system, they said, "Just give me five grand a month every single month moving forward." And that's the evolution we saw a couple years ago. Now predictions and where I think things may go, we're looking at security becoming a lot different in the tools that these bad guys are using, from voice recognition, meaning that there's examples of attacks that are happening where they're actually getting into a system, recording phone conversations and figuring out the CEO's phone messages and the way he talks and recordings and then building a talk track and able to through their technology call somebody as the CEO, spoof their number.
Maybe even send them an email and say, "Hey, I just sent you an email. It's about a wire transfer. I need you to send this money over." I found two cases of that actually happening already. So, there are a number of different things that we're looking at saying, "What's next from these guys?" and we even see some of the funny spoofs where it's the facial technology where they're building out the - You've seen examples of Tom Cruise that are out there online where he's saying some wild stuff, but it's somebody that made this look that way. What is that going to look like from technology? Or maybe it's video conferencing, a Zoom application where that may not be the person that you think you're talking to. That's what we're looking at five, 10 years out from now. It's a frightening place.
Rahul Mahna: And I think the term is deep faking.
Eric Torres: Deep faking.
Rahul Mahna: Is that what you're going to?
Eric Torres: Yeah, that's what I was looking for.
Rahul Mahna: So then maybe you or Dan, as the evolution keeps going, where does backups go? Are we still just going to be backing up files? What are we backing up?
Dan Ciccone: So your multilayered approach is the key to this to begin with. This is the last resort. You've already been hit at this point, but you need to understand, "How are these attack vectors coming in?" first of all. Where does your data live because it's not just on network these days? And that'll be on having - SIRIS has multiple tools to protect SaaS application data, workstations, laptops that leave network consistently. Where will backups go in the future? I love the hybrid approach. I think that more often than not people say, "Hey, why don't your backups just go directly to your cloud infrastructure?"
You have to consider disaster here and I think that things will remain the same for quite some time. There's going to be a shift in the cloud with infrastructure as a service and not having servers internally anymore, perhaps building them in Azure and things of that nature, but we're already starting to play in those realms as well. We released a product that protects Azure virtual machines. I like having something onsite. This allows for quick recovery internally, but in the event of a disaster, true disaster, let's say you lost everything and now you need to recover from cloud, the time that it takes to bring data onto a system or download it and then restore internal, that puts people out of business.
There was a Gartner report when that was run when I first started working here that said, "Well, over 75% of businesses would not recover beyond a certain threshold of data." If I can have you running in our cloud, simultaneously copy the majority of the data onto an appliance and then send this overnight, you turn off here, you turn on here, you restore while you're running from here, you're still talking about minimal downtime in the event of a disaster. So I think the shift will be, "Where do my systems live? How am I protecting them?" but we need redundancy. You always need that other copy somewhere else. That's the big point.
Rahul Mahna: That makes a lot of sense.
Eric Torres: And if I could jump in some prediction, some forward thinking things in the next couple years, the world shut down last year. Everybody wants to go work from home, work from the cabin, work from the beach house. We're looking at and saying, "We're backing up a device. We're backing up a server. We're backing up a workstation. What if we back up a person?" No matter what device they're on, we know through how they're authenticating, how they're getting in, whether that's through their phone, through their laptop, through a desktop to their kids computer, maybe they're in a kiosk somewhere and backing up that person's data no matter where they are, that's what I think is years down the road, but looking at backing up that person.
Rahul Mahna: So we have a couple minutes, I see a couple of questions that keep coming that are very similar. What are your gentlemen's thoughts on Microsoft? So we get a lot of clients that say, "I'm in the cloud. I'm on 365 or I'm on the G Suite," we can take each one, they're backing it up, "Why do I have to back it up?" They're already taking care of my 365. What is your reaction to that?
Eric Torres: The amount of retention that they have. They only have a set retention model. And in today's world, we need to keep as much data as we create. Microsoft's retention model, I believe it's 30 days, depending on your subscription model. After that day, if let's say you accidentally delete a file or accidentally delete an email, it's day 31, it's gone. They don't have it. This keeps you in control of that data if you have a second copy of it living out there. And also we see a number of that shift of moving to Teams and SharePoint Online and moving over to the Microsoft cloud applications.
That just introduces more opportunity and vulnerabilities. Ransom cloud is a very real thing. Now there's attacks that are happening within Microsoft's data centers and locking down the data within that data set. And that just proves even more you need that second copy of data to recover and keep working in the event of some downtime.
Dan Ciccone: I'll piggyback off of that as well, Eric, to saying that there is a shared responsibility model and Microsoft is not responsible and it's right within their SLA for accidental malicious deletion, ransomware, malware that may hit the environment. They're there to keep their infrastructure up and running and they do a pretty decent job of that, although it does go down from time to time. So with that said, we have another product outside of SIRIS called Datto SaaS Protection. It's an exact copy of mail, calendar, contacts, OneDrive, Teams and SharePoint data. It's brought in cloud to cloud, so no software in - No hardware, excuse me, in this solution and it's a very simple recovery process as well. You're talking about, again, the same platform, everything being built on ZFS and Datto Cloud, point and click restart capability within seconds of accidental malicious deletion.
You do bring up an interesting point that ransomware cannot only hit the OneDrive or Team data that they may have local on their computer and then go off and say, "Mail can be hit with ransomware." We showed an example of this at one of our national conferences.
Rahul Mahna: It's a great point. It's in their SLA, which is a really good point-
Dan Ciccone: They literally tell you.
Eric Torres: They tell you to back up.
Dan Ciccone: You need a third-party backup of this data.
Rahul Mahna: It's fantastic. Well, thank you, everybody. We're at the end of our time. We will respond to all your questions. We hope this was informational, and if you want any help in measuring and managing that risk, reach out to me, reach out to our team. We're doing a wonderful job with our partners to try to provide these solutions to you. So with that, I'll turn it back to you, Lexi, and thank you, everybody.