OpenStack Podcast #19 Yuriy Brodskiy
Symantec’s Director of Cloud Platform Engineering, Yuriy Brodskiy, was a really interesting interview–not only because he was a very early adopter of OpenStack in his PayPal days, but also because he now works for one of the pioneers in software security. He gave us some surprising insights into how his company views open source in general and OpenStack in particular, as well as what they’re doing to make the cloud more secure. He also discussed:
- How cloud changes the culture of an organization
- How OpenStack changed perceptions of open source software
- The upside (and downside) of rolling your own distro
- The future of PaaS and containers
- What Symantec is doing with OpenStack
- What to consider in order to create a truly secure OpenStack environment
For a full transcript of the interview, click read more below.
Jeff: All right, good morning everyone. I am Jeff Dickey from Redapt.
Niki: I am Niki Acosta from Cisco and we have a really awesome guest with us today. Yuriy, introduce yourself.
Yuriy: Hi, everyone. I’m Yuriy Brodskiy. I am the director of engineering here at Symantec.
Niki: Yuriy, we’re really excited to talk to you today. It sounds like you guys are doing some really cool stuff with OpenStack, but we typically start the show by asking you how you got into tech.
Yuriy: Sure. Not a very exciting story, but start my [inaudible 00:00:41] year in QEA and spent a couple of years in different companies doing quality engineering and, ironically, now, I ended up at Symantec many years ago where I moved into development and a couple of years there on a website and I got a call from a friend of mine to join a small company called PayPal. I moved to PayPal where I spent a lot of years. Over 11 years at PayPal wearing many different hats.
During PayPal, to actually do quality automation there and build some processes, and through all those years I move all the way from QEA and somehow I ended up in Ops, and then what you have, but the last seven years of my PayPal career I was wearing different hats in operations from doing L-3 support to engineering. Here I am, the second round at Symantec, now.
Niki: In those early days, too, I remember just as cloud was starting to hit critical mass. Prior to that, Ebay and PayPal were kind of leading the charge in terms of cloud. They were an early explorer of OpenStack most certainly. What was that like adopting cloud in an organization before cloud had really hit critical mass?
Yuriy: It is an interesting journey there so Ebay was earlier adopter. Right? Ebay built before there was an OpenStack. Ebay built their own cloud platform, and that adopted OpenStack, actually before PayPal did, but what really on the PayPal side we knew we need it, but I don’t think we actually answered how to. Really, until [Surram Mandear 00:02:47] some of you might know joined PayPal. He joined us from Ebay. He came in and started working for Surram and he said, basically, “Look. We’re going to move PayPal in the cloud and we’re going to use OpenStack. My first reaction was go to Google and figure out what he is talking about.
What is OpenStack? Right? Nobody really knew at PayPal what OpenStack is how to best solve our cloud vision. That was just when ESX came out and, now, looking back, it’s kind of funny, but, back then, when we looked at ESX we said, “Look. It’s pretty stable. It’s a good platform. Let’s go. Let’s go build it.” The story is there. It was an innocent adoption because I am pretty sure like with any company moving traditional IT processes into the cloud world it’s not an easy journey. There is a technology aspect. There is a people transformation aspect, but I’m sure some of you heard PayPal’s story.
I mean, PayPal had been fairly successful in that journey. Probably, but being a little bit more aggressive than some other companies by going directly into production versus starting with labs and POCs and QE environments. We are taking a very similar approach actually here at Symantec just saying, “Look. We know what OpenStack can and cannot do.” We’ll solve the real use cases in prod, and then when the development requirements come in, we’ll solve them as well.
Niki: How did you guys deal with finding the right people to be able to implement and run cloud at that time? I can’t imagine there were a ton of people out in the market. You were like, “Yeah. I’m a DOT engineer or I’m a OpenStack Python developer or engineer.” Did you guys just learn along the way?
Yuriy: We have. Part of what, I mean, we had a bit to jumpstart. We had a big help from Mirantis. Mirantis came in and they helped us to jumpstart the effort, but to your point, there was no people who actually knew what OpenStack is. There was no cloud people. There was no devops people. If I look at our original team when we started building a cloud team, it was me who had to Google what OpenStack is. Our first engineer who is now our [inaudible 00:05:33] here at Symantec right now. He is our normal guy. He was a Hadoop engineer. Right? We hired a Hadoop guy who just was a very good developer.
That’s how we really built the team, not by let’s go hire OpenStack guys, but really just hiring lots of smart engineers who wants to go solve real problems.
Niki: How are you doing with that now? I mean, obviously, I can imagine but like every other company on the face of the earth. There is still a pretty strong desire to bring in cloud talent. How are you managing that now at Symantec?
Yuriy: It’s the same, Niki. Maybe it’s actually harder because back then it was people were not jumping the OpenStack bandwagon so it was much easier. Now, you’re competing and especially here in the Bay competition is huge. It is very tough. It’s very tough to find good talent, but, honestly, we’re taking the same approach. If we find people who know OpenStack great, but really just finding great engineers who want to solve the problems. They’re learning the way.
Jeff: Yeah. That’s good. Hire smart and they’ll figure it out.
Yuriy: Pretty much, unless somebody else has some other magic. If there is, I’d love to hear it, but I haven’t figured that one out yet.
Jeff: What were the drivers behind the decision to go with OpenStack early on in the ESX days, and at PayPal?
Yuriy: Back then, if you look at PayPal infrastructure, as I said, we were physical world, paper and digital. We did have some virtualization on VMware and we’ve done all the MSCs. Right? KVM and bunch of short scripts, writing scripts. It’s all good, but we knew we needed something else. The reason really wasn’t let’s fix it for Ops. The reason was let’s fix it for dev in a sense of let’s fix for agility. Right? They need to move fast.
For the whole cloud effort, what, actually my job I was running the team responsible force solution engineer, which, basically, the guys would work with developers, figure out what the needs are, and then go build it up on the production side. Right? Build the servers. Build the network. Build the [inaudible 00:08:09] etcetera. It takes time. You can imagine the business world. The world is changing and quite a bit of demand of getting our developer a bit more agile and be able to release. Everybody was looking at Googles and Facebooks of the world or even the older fancy blogs saying, “Hey, they can kick butt. It’s so fast, so cool.”
Of course, your executive team looks at you the same way so what are you going to do? Right? How are you going to solve it for me? Realistically, even though, yes, it was ESX, but when we looked at the option the community behind OpenStack and where OpenStack was going it was quite our best choice of just rally behind it.
Jeff: Yeah because I think there was a lot of choices back then. I mean, you had CloudStack and Eucalyptus and you had OpenStack.
Jeff: Do you guys look at them and make the decision or was it an easy-
Yuriy: We [inaudible 00:09:13] we have not. Right? I mean, we knew what they are, but we just basically went heads down and just said, “OpenStack it is, and we’ll make it work.”
Niki: Yeah. At that time, I’m not sure that some of the other options were open source? Right? I mean, at that time I think they were still proprietary solutions?
Yuriy: To a certain extent; I mean, some of the principles that we adopted is no vendor logins. That’s what OpenStack provides. It’s open API. It’s you are free on your hardware and it’s huge. It’s a huge benefit. I mean, again, if we’re 24 percent on development side it’s one angle, but you do have to, at some point, start looking at the threshold costs and having the folks [inaudible 00:10:03] operational side is huge as well.
Jeff: You think from-
Niki: Yeah, totally.
Jeff: -working with your doing OpenStack at Symantec now and you started pretty early on, do you think OpenStack is now easier or more complex from when you started?
Yuriy: Good question. If I would take … It depends on the angle you look at. Right? If I would take OpenStack as it is…say Icehouse or Juno, and go back into ESX days, I would say, “Hey, yes. It’s much easier.” It actually works, but we’re not then. We’re here and requirements are different because there is expectations. There is a level set and certain things that were a wild factor a couple of years ago are … It’s a minimal level of expectations today. You take that and you increase complexity by requirements.
People no longer interested in spinning up VMs. It should work. Give me something else. That’s where complexity really comes in. It’s stitching it all together and solving the real use cases, not just virtualization use cases.
Jeff: Yeah. I like that.
Niki: You have your core projects. You’ve got whatever, Nova. You’ve got Cinder, Glance, Horizon, whatever. Have you implemented any of the ancillary projects or are you exploring any of those projects especially like Ironic or Sahara or Savanna? I forget what it’s called now.
Yuriy: Yeah. We are, Niki. The core is there, of course, you’ve got to run. We are, in answer to Jeff’s question is it harder or easier, and that’s why it’s harder because the requirements are bigger so, for example, at Symantec we’re looking at [inaudible 00:12:04]. We are Symantec. We are security. As you can imagine, this is near and dear to our hearts. Then we’re looking at pride and working on pride such as [inaudible 00:12:15]. Symantec has been very involved with [inaudible 00:12:19] effort as well.
Ironic is definitely in our roadmap for probably we’re going to start looking at it very close in the next couple of months for the same because, now, all those capabilities comes in. What it does allow us to start removing our custom and homegrown scripts. Right? Ironic, specifically. Talking about bare metal provisioning where we had to build our own system to do hardware and bare metal management. The hope is that we can solve it now with OpenStack standard APIs and start clawing away our own craft.
Niki: Are you at liberty to discuss what your hardware stack looks like and, secondarily, does the hardware matter?
Yuriy: I probably wouldn’t go into too many details, but, I mean, there is no secret. It’s generic hardware. There is nothing special about it. Data [inaudible 00:13:18]. Right? Basic to use we actually are looking at more High Def solutions right now. [Inaudible 00:13:28] performance, and, again, it’s all driving project requirements, the business needs so we have a big ask on performance so we’re looking at this is these solutions to be part of the platform, but, realistically. From an OpenStack perspective, we don’t see it as it matters, really, what hardware you run.
There are certain fields that we’re interested in terms of security from a Symantec perspective, but besides that, hardware is hardware. Right? What really matters, what we’re looking at from our side because of the scale, really, dollars matter because putting the older piece of equipment in a cloud has become expensive because you’re not going to be able to get the same compute power out of they start to break as if you can replace it with something newer so when you are running a lab this probably doesn’t even matter.
You’ve got a couple of developers there spinning up VMs, but you start going at a quite larger scale and different use cases. For us, it’s really important to actually have top performing equipment, part of the cloud.
Niki: As far as OpenStack goes, it sounds like you guys have gone the DIY route, maybe, but are you guys relying on any commercial distributions of OpenStack or are you guys rolling your own?
Yuriy: We are rolling our own. We are not. We’re taking the same approach as we did at PayPal. It’s basically no logins. Part of our goals here is self sufficiency so we are not relying on any distribution for that.
Niki: If the classic enterprise VP came to you, VP, a cloud or whatever, and said, “Hey, Yuriy, we’re thinking about implementing OpenStack.” Would you tell them to roll their own or would you tell them now to look at commercial distributions knowing what you know about how long it’s taken you to get where you are now?
Yuriy: I would probably say it depends on the use case. If the use case is … I mean, I wouldn’t call it simple, but it’s not very complex. We’re trying to solve for that. We’re trying to solve … It’s a chicken and the egg, Niki. Right? I mean, the reason we’re building our platform, our team here, because we are here for the long run. I do not believe that in the long run you can afford because, I mean, OK. Fine. OpenStack is open source and that’s one of the biggest drivers, but if you’re basically going back and logging yourself in at the vendor, what’s fundamentally different? That’s my take. I’m not saying if I’m right or wrong, but it might be those.
Jeff: Yeah. I don’t know if you guys saw the article that Randy Bias did recently talking about-
Jeff: -the distros. I think that was interesting, but, yeah. It is one take and it does matter in your requirements and what level of support. What do you do or do you have any war stories or recommendations around upgrading because you’ve probably had to upgrade several times?
Jeff: What does that look like?
Yuriy: I have war stories and I have success stories so we got both. Right? There are definitely war stories. Actually, here we’ve done pretty well at Symantec. At PayPal, again, we were early days. We were learning. We didn’t have enough experience on that. I’m sure the team is doing much better nowadays. It was painful. It was painful because you run your website. You can’t afford taking any outages because then it impacts that’s the bottom line. Right? Business doesn’t really care if you’re in an OpenStack meeting. We are a physical first structure. Uptime is uptime.
Not without failures, right? We had that, but, again, it was early days. I’m not going to say. I mean, there is big discussions going on right now on the forms and [inaudible 00:17:51] this or how are we going to do zero downtime upgrades. It’s very important, especially for enterprises. We just actually finished our upgrade here at Symantec so here at Symantec, basically, one of the rules … Well, I don’t know if it’s a rule or policy. Whatever you want to call it; guiding principle, let’s put it this way. We have these when we’re staying maximal one release behind.
We are on Icehouse right now. Once we come back from Vancouver we’re going to get in Juno, which means we upgrade every six months. That is an expensive exercise, very expensive I think. Yeah?
Niki: Are you at liberty to say how many people you have working on this?
Yuriy: Sure, no problem. On the upgrade side, really, we’re talking about … Let me attack that differently. The way we have a process is the engineering team, the guys who work in the OpenStack or they are the ones who basically go in and preparing all the upgrade scripts with all the testing, but then our Neutron guys are the ones who actually execute. On the infrastructure side that’s actually is going to do all the project changes and configurations and actually all the collaborating side. I think it’s four to five guys. We just done our last upgrade.
I think they submitted a document for one [inaudible 00:19:28]. We upgrade our site from Havana into Icehouse with zero downtown on the data plane, and I think it was just under 10 minutes on a controlled plane of OpenStack.
Niki: That’s incredible.
Yuriy: It is. A lot of planning, but at the end of day, it was very successful. You go from bad to better. I guess you learn. You learn what works, what doesn’t, and we are where we are. Now, we’re shooting for getting to five minutes once we go into Juno.
Niki: One of the things that we talked about prior to the start of the podcast was how cloud is actually changing the culture and processes of how stuff gets done in some of these organizations that have adopted cloud. It’d be really useful to get your take on that as someone who’s done it, actually.
Yuriy: No, absolutely. It’s very interesting, Niki, to see. At Symantec, it’s a very new team. The cloud platform engineering team is a brand-new team at Symantec. It’s what we call a startup within and, sometimes, it’s an overloaded term and that was the sales pitch that was given to me when I joined Symantec. “Hey, why don’t you start up with a very well financed company?” Fine. I hear that spiel all the time, but reality is it’s actually somewhat true, and it isn’t for is we build this and we will build into this team who is adopting truly OpenStack to our hearts.
We are really taking most of the OpenStack processes such as CI pipelines and documentation, and process for reviews and blueprinting, and actually there is communication and manning and IRC, basically, they are throwing away all this traditional enterprise processes and it is, look. We are OpenStack. Right? We are embedded in OpenStack. It’s fairly successful with such a large community around the world. Why not … try to invent the wheel? Why not just follow and take the best there is? What really happened is Symantec is going through transformation.
A lot of people we’re building the platform so they adopted the platform. The teams are moving to the devops world and the question becomes, “What is the best way to do it?” There is no silver bullet. I mean, we all know that. Devops is a loaded term, and everybody is doing it their own way, but really the message and what I’m seeing from our business unit looking at our teams. We are working in OpenStack and we warn them through the processes that we follow. They are really OpenStack flow. It was already scheduled and how do we do the documentation? How do we do designs?
It’s the same model. You put a blueprint in. You get [inaudible 00:22:57]. It gets approved. You buy documentation you put it in gear and it gets approved. We have the same plus one, plus two, minus one, minus two processes. It’s very straightforward once you understand and it’s very powerful and I see a lot of eye openers on the product side when we are saying that makes sense. Why don’t we move our process and to follow the same methodology OpenStack does? If you have a question look at OpenStack, that has been our default answer to a lot of businesses in Symantec.
Niki: Obviously, there is naysayers in that process. How do you deal with the naysayers? That’s always the trouble when you get smart engineers. It’s like what do you do with the naysayers?
Yuriy: It’s always going to be like that. I mean, just like I said, there is no silver bullet, but you go with what you got and, like I said, now, we’re not pushing it to anybody. It worked for us and the same things for that is something that we’ve learned, but I see it as a huge, what do you say, it helps jumpstart a lot of processes that stems more from traditional waterfall processes into more agile, devops cloudy world.
Jeff: What are some of the gaps you’re seeing right now? You’re talking about some where OpenStack kind of fits. Where are the open opportunities or where do you have to look outside of OpenStack right now or do you have any kind of a wish list?
Yuriy: I do have a wish list, but that is, again, it’s an enterprise versus smaller scale because with an enterprise you have all the enterprise use cases. It may not necessarily be OpenStack specific, but you do have to manage your hardware lifecycle management, but your hardware, having the onboard. You do have to … Yes. We’re looking at Keystone so we have our Hadoop infrastructure and something else, and how do you manage your identities, compliance security, all those things? I mean, they’re not necessarily OpenStack-specific issues, but once you put this platform together it all has to come together because it needs to be …
For operational efficiency, we need one ecosystem. It’s not just getting gear. Somebody builds the gear and then here is OpenStack. It’s not magic. It’s a platform. That’s where we’re doing a lot of stitching yesterday to Randy’s point is OpenStack from its source it’s one thing, but to build a real platform, build a quote-unquote cloud and enterprise, everything else needs to come together, your charge backs, your metering and monitoring and your alerting all those capabilities needs to be in, in order to run a successful platform.
Niki: It seems like you guys are kind of ahead of the curve probably more so than most companies I talk to regularly. What is the future of PaaS and containers look like for you guys?
Yuriy: Good topic; good question. We are looking. We are looking at it right now to be honest with you. We’re looking at different options. When we get [inaudible 00:26:38] is that going to fit? We are looking at do you have your fact sheet NASA’s projects or you have cloud [inaudible 00:26:48] in the world. What I’ve seen which is very interesting is that you go back a year ago before the whole Docker revolution, whatever you want to call it, and PaaS introduction was quite tough because you have the right application for PaaS so I haven’t seen it together too much.
I mean, some do better than others, but still. These containers what I see here right now is that our developers just to say, “Look. We are developing in containers. It works. Why do you have to change the way to do it? You’re making it worse on your platform. Is it PaaS? Arguably not, but they expect a light sort of … It’s a light application lifecycle management. That’s really what they want. They still want the same floor of here is my code. It went through CI. It went to production environment, and then somehow gets managed and then hopefully dies.
Now, manage that for me and that’s what traditional PaaS would help with, but, now, having containers they still have to manage it for me, but, not necessarily lock me down to whatever process you want. It’s a lot of push and we aren’t doing it on our side. Here we have our bare metals. We have our VMs. We do have Alex [inaudible 00:28:19] and Docker as part of our platform. We have [inaudible 00:28:24]. We have Kubernetes, but, again, the challenge is that now you are going too wide and it become hard to manage. There needs to be something that still stitches it together. Is it OpenStack? Maybe. Or something needs to be on top of it. [inaudible 00:28:43] really need to [inaudible 00:28:45].
Niki: Totally, yeah. It’s like the whole Puppet versus Chef versus Ansible versus SaltStack.
Niki: Definitely, I mean the Rackspace there were groups that used all of them. How do you rank [inaudible 00:29:02] streamline processes when everyone is using different tools? How do you keep developers happy and keep them wanting to stay where they are and writing good code?
Yuriy: It is because, I mean, if you think about … I’m not sure how other organizations do it, but if I look at our platform and here is Symantec, yes. We are part of Symantec. Yes. We are solving for the business, but at the same time, we are internal private-public cloud, if you will. There is a fine line of the governance where we can control certain things, but other things we can’t. When VM comes up … I’ll give you an example. In our initial deployment we had VMs that would come up and they will run configuration management to configure when they’re under managed. A standard kind of exercise.
Guess what? Our customers come and say, “Why? This is my VM. What are you doing? I don’t need you there.” Build it up. Make sure it works and go away. Now, hey, we’re not a public cloud. We can’t really let you do whatever you want. We need the governance. We’re a security company, but their point of view we need agility. Why are you forcing us to do something? The beautiful part about containers, now, we can have what we need in terms of controls. Yet, developers can have their part of the agility to move their port in and out without us being in the way.
Niki: Are you finding that your internal users of this cloud platform that you’re building are well familiar with writing applications for resiliency and redundancy? In other words, writing for cloud versus writing for say a VMware ESX environment?
Yuriy: We’re not there yet. It’s part of the journey. Some things are better than others. We’ve still a ways to go to write a real cloud application. That’s part of the challenge because people read what cloud is and they expect that cloud will do the magic. It will give them this … It will solve. Just put it out there and it will scale. It will make sure it works. Where cloud is the other way around. It will break. Build your apps to make sure it breaks.
My favorite thing is one of our architects here when we go talk with internal customers and we talk about their vulnerability we always tell them, “Build for two nines. And they say, “What do you mean two nines? You can’t have a platform that is two nines.” Well, the platform is not, but when you build your applications, you have to assume it will go away, but there is a ways to go.
Jeff: That’s right. Yeah.
Niki: We have a large customer at Metacloud AKA Cisco I guess that intentionally walked up to one of their servers in one of their cabinets and just pulled the plug out just to see what would happen. It’s like, man, it was good. It was good for us to know that our stuff was resilient enough to handle that, but at the same time, that’s like you got to plan for that kind of stuff both at the OpenStack layer and at the application layer, and at the switching layer, and at every layer you can possibly think of if you’re going to provide any kind of SLA to anybody. How are you guys handling HA?
Is that something that you sorted on your own or have you looked to best practices elsewhere to adopt some of those best practices?
Yuriy: Both. So definitely looking. I mean, there is a lot of good things been solved in the community, but a lot of things are our own and I guess one of the good things the guys who are part of this team they lived and breathed operations for years before cloud was cloud so they do understand HA. We are approaching from multiple layers. From physical hardware infrastructure you make sure you have your redundancy there and switching layers. From OpenStack perspective we’ve done some changes, added a new schedulers to make sure that our VMs are distributed across different full domains.
Again, to your point, you’re kind of moving up the stack as much as you go on that layer. We have our load balancers in front to make sure that we don’t have single point of failures and controllers deploying in multiple zones, but, at some point, applications will still be only more aware of what it’s doing.
Jeff: Yeah. I got two questions for you. First, are you looking forward to live migration or is that just cheating or bad architecture? Is that going to help you guys?
Yuriy: I don’t. Our customers do. And because it actually works. I had always an excuse. “Hey, go figure it out.” They expect it to work so I think I am to a certain extent actually have it solved, but it just means more work for us. It will be good for our business, but I think the customer demand is more than our role.
Jeff: The second one is how are you guys doing with Neutron? Is it meeting all expectations? Is it something that you are writing a lot of stuff towards or have you skipped it altogether?
Yuriy: We are not doing anything with Neutron. We’re writing on Open Contrail. Again, Symantec is security and it’s kind of interesting coming from PayPal security, but it’s a little bit different level here. We are a security standard and we need to make sure that there is nothing else more secure than our cloud. While three is a little more detail and [inaudible 00:35:22] around those features from the get-go we said we’re going to build SDN. Our cloud will run on SDN. There is nothing is going to be [inaudible 00:35:31] networks. Every single packet will go on and we’ll solve it with that. We certainly got solutions. How do we provide SDN?
Actually, the guys talk on that in Atlanta and Hong Kong. We look at different vendors. We look at different options and we blend it with Juniper and Open Contrail project and, as you know, there is not much on Neutron there. We can’t complain on Neutron right now. Ask me that a year ago when I was at PayPal, maybe we have a different conversation, but right now just not having a problem by avoiding it.
Niki: It still seems to be one of the components of any OpenStack implementation. I know that there is a lot of pro Neutron folks and then a lot of anti-Neutron folks, and people who don’t think it’s ready or people who think it could be ready with the right plug-ins or whatever. It’s interesting that you guys have gone that route, but it sounds like if it works for you, why not?
Yuriy: Look. It has its pluses. It’s got its minuses. We can argue for a long time what works or doesn’t work in Neutron. From Symantec’s side, we just didn’t have time to argue. We had to build or we had to solve the daily use cases for the company. We knew the limitation of the Neutron. Not going with Neutron we definitely lost some of the capabilities, but we have to do it. Open Contrail has been a real big help.
Niki: What are your use cases? What are you doing with your OpenStack cloud? What do you intend to do with your OpenStack cloud? What is Symantec doing in the way of innovation on your OpenStack cloud?
Yuriy: If you look at Symantec … Oops. If you look at Symantec, Symantec has been around for a long time. When people think Symantec, for some reason actually with myself included, from a long time when I hear Symantec I hear Norton Anti-virus. Why does Norton Anti-virus need the cloud? It doesn’t make any sense. Symantec is a little bit more than Norton Anti-virus. While it is a big component on the portfolio there is a lot of different capabilities and balance it has. One, Symantec should be a data company.
As you can imagine, for all the security and threat analysis it’s all about big data. It’s all about an audience. It’s a huge component for our platform, and then from the other side of the world, if you look at Symantec, a lot of businesses are still in the physical appliance model. Here is the server. Go plug into your network and Symantec will do the magic. There is not many more data centers left in the world or will be left in the world in a few years. As everybody moves into cloud, that’s where Symantec is going.
Symantec is going into cloud. Symantec will be a security company in the cloud, the best security company in the cloud. If you look at OpenStack and why we’re building this platform, is, one, your standard use case. How do we get developer more agile? Who do we get all the speed and all the good stuff? Then you get your another side of the story is how do you get your operational efficiency running the same infrastructure? Everybody it’s very similar stories through pretty much everybody who deployed OpenStack, but the next angle from Symantec perspective is as we’re building the security products in the cloud, it’s as a service kind of model.
Where do we host it? Whether you’re on it you have to sort of eat your own dog food. Make this cloud is secure as well. It’s all those three different angles that are coming in and driving this cloud deployment for Symantec.
Jeff: What’s the Holy Grail for securing an OpenStack environment? I mean, are you securing the hypervisors, the VMs or the workloads or the application itself? Where do you recommend folks look at that?
Yuriy: I think it starts with your deployment strategy. You are as secure as your weakest link. It’s really important to understand when you… before you put this first server in the data center and deploy the first piece of OpenStack. What is it going to look like? What are your security policies? What are your access controls? What does your network topology look like? Where is your provider networks where you manage your networks? Probably you are aware SDN is going to be core because, basically, we moved all the traffic up into overlay so we can have our management physical network isolated from the guests.
This is the core. This is part of the platform. Of course, and from our side we’re looking at hypervisors. How do I lock it down? Because even though you have to run your Ubuntu or you are Santos or whatever… It’s not a Linux box. At the end of the day, from my view, I look at hypervisor as an appliance. It has one function and one function only. When we’re looking at it from Symantec so how do we secure and lock it down? Strip down that operating system to minimal to what it’s supposed to be doing. It has only one function to function as a hypervisor.
There is no reason to run a full-blown operating system on it. Also, what it means is once you have less vulnerabilities, you have less patches to apply. You have much less security risks. That’s another angle that we’re definitely looking at and executing on. Then if I zoom more on the stack, Keystone. We are doing a lot of trend modeling here. We done a trend modeling and for Keystone and working in trend modeling for Nova right now because we need to understand from Symantec perspective what are the threat factors.
Where can attacks come from? Securing the core is one thing and it’s quite important. Developers must learn to write security first when they write their code, but then it’s your deployment matters. That’s really what’s important.
Niki: Is cloud secure in your opinion? Public, private, in general?
Yuriy: I don’t know. Tough question. Is cloud secure? It depends I guess. I don’t know. I don’t know. I don’t have the answer for that. I think it’s secure enough. It depends. I think it’s I see the cloud as the same as back in the day when virtualization came in. People said virtualization is not good enough. It’s not secure. It’s not compliant. That all worked. Then the same conversation started with KVM. Is KVM secure and complaint. I think it’s a journey. It’s a confidence level and maybe with some luck we’ll get technology to put in place. It will be.
Even if it’s not, it might be secure from a technology perspective, but it’s probably not secure enough from a comfort perspective.
Niki: You hear a lot about … Go ahead, Jeff. Sorry.
Jeff: OK. I was just curious about drawing the line between open source with proprietary and working out some of these projects. Where do you see that? Where would you contribute to fixing security patches in the community versus having a product that you use to create revenue that fixes that security? Is there a clear line or is it just a tough one to navigate?
Yuriy: You do mean on the Symantec perspective, I’m assuming. Right?
Yuriy: Look. I’m not a business guy. I mean, they’ll figure it out, but I’ll tell you the message that I’ve been when we’re having these conversations. My take on it is OpenStack has been … I look at OpenStack and part of the success because it is very open. It does what it does. It has a very rich plug-in architecture that all the vendors can while they’re making OpenStack better they can still monetize on that. There is no reason why security cannot be solved in the same way. Symantec has a lot of great products, but, again, it’s all been built before the cloud was there.
There is clearly no reason why OpenStack cannot have a security component in there and then if you have your Symantec or some other vendors can assume it any other way as, I don’t know, mutual, of course, better than, but you know what I mean. To apply this, if you will. That’s going to be my take. I don’t think it’s … I mean, it isn’t proprietary, but if people take OpenStack at heart and they take it as it is an open source. It’s open API and it gives you no vendor login. Proprietary might not be a long term solve.
Jeff: I see the proprietary part around security, though, is a value where you can’t really pick apart what’s going on. I mean, you kind of want that proprietary plug-in to be watching.
Yuriy: No. You have a proprietary plug-in, but what I’m saying is the flow should be generic. If you have a project that does X from a security perspective, then yes. You have a proprietary plug-in for Symantec and Symantec will do its magic. That’s how I look at this certificate-wise or whatever you want to call it. You look at LBaaS. Why is it any different? I could have LBaaS with one vendor or another. The capability is the same. If I’ll tell you if I had a capability I want OpenStack to say make sure this flow is secure. That is just an OpenStack API.
What happens on the back that’s a vendor solution and it’s up to you to make sure to see that which vendor is better to solve a case. Hopefully, that’s very interesting. Being Symantec, but that’s my way to look at it.
Niki: Being close to the initial founding of OpenStack at RackSpace. That’s the whole intent of why OpenStack was founded. I mean, there was no standard. There was a ton of lock in and proprietary solutions out there. I think for RackSpace they wanted to level the playing fields and let people sort it out. I think the community is just looking back to what was in Jim Curry’s head back then and to what it is today. I mean, he hit the nail on the head. I don’t think anyone ever thought it would be what it is today.
It’s nice to go into a conversation and people have actually heard about OpenStack and done a little bit reading on it versus even a year ago or two years ago no one had any clue what it was. What’s exciting for you? In the community, in the innovations that you guys are seeing or the agility you’ve been able to gain internally, what’s getting you excited about the future as it pertains to OpenStack and the cloud and Symantec?
Yuriy: It’s I guess the reason I joined Symantec is making a difference. I’ll kind of give you the spiel a little bit. When I was at PayPal, and we started OpenStack and the whole effort, I really looked at it like, “Look. This is a very unique opportunity.” You get it once in a lifetime. You have this new trend of OpenStack. The changes in the world and then you have PayPal who is a big company and you have to sell it at scale, very unique. Then there is Symantec. Symantec is big, but the use case is harder.
The scale is larger. There is security aspects of that. Solving that is quite an amazing … It’s a very satisfying experience because you do see immediate gains. You understand how you help and that’s the bottom line. You see actually developers being happy about it. That is a good thing. You’re making somebody’s life easier. Like I said, one of those unique opportunities. I think we’re all lucky that we are part of this amazing community that’s changing the world.
Niki: Do you think that OpenStack’s model just in general has forever changed people’s views on open source in general? Do you think it’s made an impact?
Yuriy: I think so. I do believe so. You go back five years. I mean, yeah some people used open source. I mean, everybody around in analytics, but for some reason it wasn’t really considered as open source as all OpenStack. It was just, “OK. Linux is Linux.” What I see today when we are looking at the cloud the selections what we do here for example at Symantec, and, again, Symantec is a security company that’s very aware of, “Hey, what are the risks?” The first thing we look at is open source. It doesn’t matter what problem we’re trying to solve.
Do we have to build our CNBB solution or we need to look at, well, we have our own, but deployment models, whatever that is. Whatever the problem, configuration management, whatever the problem we’re trying to solve it’s open source first and I think part of that is people looking at OpenStack and seeing, “Look. It works. The model works.” It’s very successful. You have all these thousands of smart people helping you. You’re never going to. I mean, back to our conversation in the beginning. Where do you hire people?
You use OpenStack. You just hired 5,000 developers. You can’t do that. It’s been a big impact in my opinion on enterprise, the whole OpenStack model.
Jeff: Yeah. What do you recommend? We have … Sorry. Getting the echo here. What do you recommend for folks, especially in the enterprise? Born on the Web is pretty easy. Folks like you figure it out, but the enterprise needs kind of a starting point. What do you recommend they start? How do they start? What are some best practices? How do you kick the tires and get started really?
Yuriy: Tough question, Jeff, because from my perspective I think we’ve been lucky enough to be an impact from the beginning. Well, not beginning, but been there for a while. I know where you’re coming from, but, honestly, I would say just growing and that’s why we have all our vendors in this community, who are doing the best practices and help companies jumpstart. To be honest with you, I think this is the way to go. It’s not learning on its own, I don’t think people … I feel people will walk away from OpenStack. You don’t just say we’re going to go figure this out. It’s hard.
It’s not a simple install that you run and it just magically works. You have to know what you’re doing. It’s a complicated product. It’s a platform. It’s not a product. It’s a platform. It’s a lot of [inaudible 00:52:30] for the use cases. You have to know this very well and to be comfortable so my personal recommendation is yes. You can read. You can go watch YouTube videos, but if you’re really serious about it and you want to kick the tires. As you said, is it kick the tires meaning those deployed-type of service? Yes. Get your sysadmin and he’d probably deploy just fine.
If it’s really let’s kick the tires in terms of, for example, put it in a dev environment or something like that, I would look at somebody to come in and help you out with something like that.
Jeff: Niki, do you have any more questions? I’ve got a few that I’ve written down from our overall talk. Do you have anything?
Niki: No. I’m trying to figure out why you’re echoing. Does somebody have a headphone next to a speaker somewhere?
Jeff: We’re both echoing.
Yuriy: No. I am just on this thing.
Jeff: One of the questions I have around best practices. Do you recommend breaking things out into smaller pods or availability settings or should it just be one, big availability zone? What do you recommend around that?
Yuriy: We’re breaking up. We are deployed in availability zones purely for availability. Look, a couple of reasons. One is definitely availability. You lose your control plane you lost your cloud. That just doesn’t work. When we build our architecture looks pretty much we build them to, I wouldn’t call them silos, but we try to isolate the environment as much as we can then it’s going from all your infrastructure, your DNS servers, your mail servers, whatever is needed to run the cloud. If it fails, it fails on its own. It doesn’t impact the rest of the cloud.
Now, at that point, it’s more where is your comfort level? How big can you go? Where OpenStack start breaking. There is still quite a bit of limitations, at scale to your neutral point earlier. You have rabbit all the goodness of not there yet. In OpenStack, once you start scaling you actually need to start breaking up one for availability and one for scale. You just can’t go over a certain limit of VM/hypervisors until you start seeing quite a bit of problems. From our perspective, from our deployment architecture, we’re out building regions. We’re building availabilities within the regions using [inaudible 00:55:08] and scale it that way out.
Niki: Are you submitting talks for Paris? Paris? Vancouver?
Yuriy: Yes. We have a few. We’ll see how many get accepted, but the team is very excited to share what they learn. It’s quite a bit. It’s quite a bit so we’re very excited. It’s very exciting to see that, again, that was your question of OpenStack and how it’s impacting the teams. Even it’s our own team is quite amazing because we are part of our OpenStack team itself. We run hackathons every week. We make sure people are actually part of the community. It’s very interesting to see the excitement of, “Hey, I can actually go and present and share what I learned with the larger community.” So we’ll be there.
Jeff: That’s great. Looking forward to seeing you there.
Yuriy: Same here.
Jeff: We’re at the top of the hour here so, Niki, did you have anything else or should we ask our final question?
Niki: How can people follow you or find you? Are you on the Twitters, on the IRCs?
Yuriy: I have a Twitter. I’m on the Twitter most of the time. Unfortunately, I don’t have too much time on the IRCs, but I’m on Twitter. The guys we have our engineering blog that we tend to keep it up to date with everything we do. We push it out. It’s all open source. Now, the learnings are open sourced. We definitely have our OpenStack engineering Symantec blog site is there.
Niki: What’s the URL on that? Do you know?
Yuriy: I will find it and I’ll share it with you.
Niki: Great. We’ll add it to the comments when we post the video.
Niki: All right, last question. Who should we have on the podcast? Two people.
Yuriy: Who should you have? Oh, man, now, you’re asking the hard questions. Let me see. I should have prepared for this question.
Jeff: I know. I forgot to ask or tell you that, that we were going to ask that. Sorry. That’s the one thing we missed. That’s fine, too. You can send us an email if you want.
Yuriy: I’ll send you guys an email. I’ve got to give it some thought.
Jeff: Yeah. Sorry for putting you on the spot there. Thank you everyone. Another great show. This is number 19. It’s been pretty awesome. Thank you everyone for listening and keep the comments coming. Anything else, Niki?
Niki: Yeah. We are trying to secure the ability to be able to broadcast from the OpenStack Summit in Vancouver if you guys are interested in being a guest for my broadcast there, let us know. It’s looking pretty good. That will do it, but I’ll keep everyone posted on next week’s show. Who do we have next week, Jeff?
Jeff: Next week is Joe from Swiftstack.
Niki: Oh, Joe, an amazing human being. Well, Yuriy, thank you so much for taking the time. We had no idea that you’d take me so far down the path and it’s amazing that you guys are doing this with just a small, sharp team, just absolutely mind-blowing. Good work there and-
Niki: -we’ll see you in Vancouver.
Yuriy: Yeah. Thanks for having me. Thanks, guys.Tags: