#28 Shopware hosting and scaling challenges with Benno Lippert from maxcluster.

Borys talks with Benno Lippert from maxcluster about Shopware hosting and scaling challenges. Considering preparations for Black Friday. Learn what’s specific about Shopware hosting, how to approach Black Friday picks and traffic.

- Borys: Hello everybody we're recording in the very specific week as we talk with our guest today. Ah this is just the week of black friday and today I have a banner from Max Custer um a hosting company. Who agreed to make the recording just in this crazy crazy week. Thank you very much for finding some time between calls with the customers who are just starting the new promotion and and everything is getting read all the all the servers nice to have you here.
- Benno: Yeah, step thanks for all the invitation and great to be here.
- Borys: I don't know if you remember we were going on one event back together in one taxi and we discussed that we should record it record it one day I think it was a few months ago. So it's great to it's great to have you here. Okay Benno. Can you say few words about max cluster what is max cluster. Um, what are experiences of max cluster in regards to shopware.
- Benno: Um, sure so Marx Cluster is a specialized managed ecommerce hoster and um, we celebrated our tenth anniversary this summer we host more than 2000 web shops mainly in the german speak in.
- Borys: Great.
- Borys: Okay, okay, so.
- Benno: Area and we live and breathe ecommerce hosting. So we really think Ecommerce hosting different from the ground up and that's that's something that's really unique from a hosting point of view.
- Borys: Yes I agree with you. We are all wetrix also are working with the hosting providers who really do it for ecommerce because otherwise if you do wordpress and some general things. Yeah, it's. Ah, it's a completely different world. It's it's hard to explain someone um or understand it. It's it's not a normal normal type of hosting call. So we'll have some good some good experiences here to talk about and in general yeah, you're you're from Germany and I have more and more guests from Germany. Ah so I'm very proud of um, ah. How do you see the shopware position right now because you you from hosting perspective will have probably a little bit different perspective than um than anyone else because you see those shops life. Yeah, you see their challenges. Ah if few hosts. Ah. Ecommerces you also yeah, host some other competitors. Yeah, like Magenta or presta shop or some other some other platforms. So how do you see put the positioning and the proposal that shopware has on the market right now.
- Benno: Shopper from my point of view has the very good um valid proposition. It's a mature open source ecommerce framework especially the version 6 and especially in Germany. Um. Shopware is gaining more and more traction and taking market share from Magento so ambitious merchants who have achieved a lot on Magento for example.
- Borys: Um.
- Borys: And.
- Benno: Replatforming um towards shopware and this is clients from all sizes and all verticals who are um, transitioning to shopware. Yeah, we don't see as many as big clients. Um on shopware as on Magento this is.
- Borys: Not yet probably.
- Benno: Um, a matter not yet I Guess it's also um, um, a matter of time because especially um, the open source solution has a very ah predictable price point and merchants together with um, a good agency or a skilled agency. Can. Really achieve a unique um shopping experience with shopware.
- Borys: True and as you and as you see from from the hosting perspective. Why customers are choosing Shopware. What would be those the biggest the biggest value um in in this in the system this that it's Opened. It allows to modify Customize. Ah or or some other. Some other aspects.
- Benno: So most of our clients um are customized clients and um, they they choose shopware over Magento for example because shopware as a company um cares about open source.
- Borys: True.
- Benno: And has a very compelling product roadmap so they are listening to the clients and are delivering new features that help merchants to really tailor shopping experiences and this is crucial in order to.
- Borys: I.
- Benno: Compete with um platforms like Multiji channel platforms Amazon and so on. So if you want to really have a striving ecommerce business. You need to be able to um.
- Benno: Come up with a unique value proposition and the the channel and the features and the depths of the of the um knowledge that you share and the experience that you create with your ecommerce shop are setting you apart. So like.
- Borys: Yeah, okay, ah, and so looking at the yeah, you probably know with the best right now in this week looking at the hosting of not maybe even shopware but in general as you mentioned you have a focus on Ecommerce platforms. What usually people don't understand what they don't consider you know when when they go with just an average hosting Provider. What is so specific in in you know in ecommerce hosting.
- Benno: Ecommerce hosting in general is really complex if you want to do ecommerce hosting and get it right? Um, you have to have to really? Um. Master many different technologies you have to gain deep knowledge of all the various technologies that ecommerce frameworks nowaday use like Redis Elasticsearch Rabbit mq varnish etc and you have to orchestrate all these.
- Borys: Um.
- Borys: From.
- Benno: Technologies for high availability because what doesn't or shouldn't happen is actually that the shop is offline or something is not working and that's ah, that's the next thing that you have to ensure is a very very good monitoring. Not only server stuts you have to have to know about more things. Um, it's not not enough to um monitor only um, standard metrics you have to ensure that the ecommerce framework is actually operating you have to. And ensure that an item can be add to a cart and the checkout is working that the response times are good and so on and so on and that that has to be done 24 hours 7 days a weeks three hundred sixty five days a year um in order to to do the sayers. So if you yeah.
- Borys: Jump.
- Benno: If you don't have if you don't have a person you can call when something goes wrong and this person isn't able to help you and figure out where is the problem. Between all these various technologies. That's really yeah, not um, not working that's harming your business and many times those problems or cure after regular business hours end. So it's was ah b two c shopping starts at four zero P M and that's ah, that's a timing where many um, many hostsers don't have um the best people on call I would say.
- Borys: Yeah, yeah, I also have an experience that a good ecommerce hosting company has to also understand the platform so be in the middle you know between um, between typical devops but also understanding what what kind of the mechanism are working internally. Ah, so to know a little bit in this case of shopper of Ph P and so on to really understand what depends on what and if if something is happening on the hosting Site. You really need to go Deeper. You cannot be just the hoster like okay we put something there and it will be working and you will not handle ecommerce because as you mentioned exactly this is too. Too complex. But.
- Benno: Exactly and when when I talk about knowing the technologies like Redis or elasticsearch you have to also know how the ecommerce framework like shopware facilitates. Those.
- Borys: Exactly.
- Benno: So weather is a session storage in Redis and if you um, try to tackle a problem you have to look deep into um the application itself in order to fix it and um, we achieve this by.
- Benno: Very skilled, um service employees. All our linux admins are also trained in 1 of the ecommerce frameworks and many of them are also certified certified chopboard developers or certified magenta developers um, in order to be able to.
- Borys: Um.
- Benno: Um, Techodos Program Problems. So Our goal is always if a ah client either agency or Merchant gets in touch with us. We try to solve the problem within the first interaction. So That's also something. Um, yeah. Which is with with um, selling stuff. Um, you you want things to be fixed as quickly as possible and even if the hoster is not the the root cause of a problem we try to help together. Um.
- Borys: No.
- Benno: With the agency to figure out or guide the agency. What's the cause of the problem is it a slow running sql statement or is it a bad um configured. Session storage or is it maybe a corrupt. Um Elasticsearch um Index or whatsoever and therefore you're totally right? We really have to um gain deep knowledge about the framework itself in order to Tackle. Um.
- Borys: And.
- Benno: The problems that are cure.
- Borys: Yeah, true, exactly so looking at the current current week. So how do you prepare for such an event. Yeah um, what do you do? How you how we prepare cle it clients. For and even like black Friday it's more black Friday week. Yeah, and after Black Friday of course, there's a high season till ah, almost the end of ah end of December so how do you approach it to.
- Benno: Um, now.
- Borys: To make sure that at the same day. You don't have 2000 customers calling like okay, we're down. Yeah.
- Benno: Yeah, yeah, um, exactly so um, yeah, what we do is um, every year we form a project group um a black friday or um, sort quarter project group. We call it. And this this group gets in touch with all our clients. So we we do a mailing and asking if they do plan any promotions or um, specific things regarding Black Friday singles day cyber Monday whatever ah, they come up with and um, the way we we do that is um that we try to provide as much context. Um for the client as possible. We have a. Ah, feature which is called shop performance that automatically scans um our client's clusters for common performance issues. So for shopware those are 35 um, different.
- Borys: Um.
- Borys: Okay, okay.
- Benno: Things that we can optimize and those um are then um, put into groups which is critical which is just ah information which is already good and we use this information to to gain.
- Borys: Okay.
- Borys: And.
- Benno: And overview how good and how healthy ah current setup is and this is something that the the agency or also the client can um, look at in our managed center which is a custom build application.
- Borys: So.
- Benno: And then um, once we do a survey ah basically with our clients. What do they plan and what are their assumptions and in in many ways The the hardest thing is to get the assumptions right. So How many traffic will I see how many cards will that be and so on because this is really crucial to come up with ah um, with the right scaling of the hardware.
- Borys: Okay, okay.
- Benno: And um also to get an idea whether the Peaks we expect will cause problems or not because if you do performance optimization you optimize for a certain number.
- Borys: Okay, so.
- Benno: So it can be 10000 unique visitors. Um and 5000 cards or checkouts in a certain amount of time and everything runs smoothly and you don't expect anything to break. But if you hit. A certain number maybe 20000 unique visitors and 10000 um cards that can be a totally different point of view and depending on how big the scale is that you expect.
- Borys: Exactly.
- Benno: We really um, have this dedicated team that consults the agency and the merchant what we can do to scale. So.
- Borys: You know, but on the other hand we have black fridayers. Not it's not like the first year so how do you see? the majority are the customers. Do they really learn out of it that. Have to prepare for it or it's still like you know a completely greenfield like oh yeah, we go with 5000000 ah you know mailings and and we spend a lot of money and they're like surprised to you see that they they're learning because of course within the time I assume the markets the better and better experience. Yeah, how to.
- Benno: Um, yeah.
- Benno: Yeah, um, so the majority of our clients because those are already most of them are ambitious clients who do know what they are doing.
- Borys: How to handle it. So how is the majority.
- Benno: And they get better and better each year I think the majority of our clients because of this um actions we take are very well-prepared but sometimes um, you underestimate, um.
- Borys: Um.
- Benno: The the traffic because maybe you land in such a dear community like my deals in Germany or something and ah then the the traffic that hits the server is just way higher than than normal. Yeah.
- Borys: Expect that? okay.
- Benno: And um, sometimes it works because you're running out of stock very quickly or see that maybe your promotion is too good, but um, sometimes it just.
- Borys: But.
- Benno: Breaks and then you need also a skilled team that's able to fix fix the problems and stand the traffic. What we also saw um last year is um that yeah that beside the real traffic. Also get um, ddosed by who who knows actually I'm I'm not um I'm not saying that it's um, competition or something but um, actually ah you know you never know in the end and those are then things that also.
- Borys: And.
- Benno: Um, come into play which which has to be taken care of so right now. Ah the the question you said asked are in general the clients better prepared or not I would say yes, but many of them. Um.
- Borys: Um.
- Borys: You.
- Benno: Do have problems to get the assumptions right? because the ecommerce is is growing and growing and um, this is something. We also try to help us to consult them. We take um, especially for Black Friday we take
- Borys: Okay.
- Benno: Um, traffic and checkouts and carts and store them on our cluster. So the the data belongs to the clients and we only use this um in order to prepare for this specific season and then do an estimation um on the on the now. Yeah data. We saw last year so we take the the months before Black Friday then the peak event and the months after and the growth that we saw last year can be extrapolated. Um this season again. So if you're.
- Borys: So.
- Benno: If your own regular traffic per day was like maybe 10000 visitors and ah the black friday peak was like 3 times as many visitors like 30 k visitors. Um, we would use those numbers um to to.
- Borys: So.
- Benno: Challenge The assumptions you have right now because um, yeah, it gets harder and harder to get the assumptions right? and see um where where's the traffic going.
- Borys: Exactly and looking with your experiences with shopware. So what are the shopware challenges like some typical issues that people don't consider or some typical you know challenges that you have to deal with ah. Um, when you get on board for example, New New shop or shop and you're getting ready for black Friday week.
- Benno: Yeah, so the I think the the main things that have to be in place is a proper configuration of all the stuff. So what we see in in many cases. For example, if ah.
- Borys: Um.
- Benno: If a client was with another hoster and they see performance issues and they fear the traffic increase during Black Friday um we see an increase in clients moving um to us and many of them have. Don't have a really good setup so the basics we see is that lots of the cache and session storage is still in the file system and not in redis so those are the things that we set up um, many many times and and get it right.
- Borys: Okay, yeah.
- Benno: And there are um, php optimizations regarding regarding the op cache for example to cache the um, the um php. Um. Statements that are recurring and used very often. That's um, one of the most common things we see so getting the opcache right and setting it up properly. That's um, one of the.
- Borys: Okay.
- Borys: Okay.
- Benno: Other things. Um, 1 thing you see in peak scenarios is that um the admin worker so the the job that runs background processes sends email and triggers stuff within shopper that's run when you work. In the admin and working in the admin and um, having the um, um admin worker set up in that way. Um, also leads to really bad performance and when. Yeah, our approach is then to configure. Ah the admin worker to be run. Um as a cle worker and um in general try to orchestrate it with system d for example.
- Borys: Yeah, cool. Ah yeah, but those are those are the type of let's say like ah deep in technical things. Um, and what what would you say would be like a 5 top. Things you can do to really increase the performance. Yeah, some some ticks and trips. Also that ah that the customers should consider during this time because Black Friday is the 1 thing it will be really a peak but following next four weeks are going to be anyway. Ah, hi yeah, it's it's not just this week and it's over people don't turn off their computers. They just ah, they just do some shopping and then later on they also still continue on on present gifts and so on.
- Benno: Um, yeah.
- Benno: Yeah, like um, it's always good to um to fix the the issues you have so the the low low hanging fruits is like configuring Redis configuring opcache um.
- Borys: But.
- Benno: Trying to ah tweak. Um Mysk Well um to get the buffers right? Um, before the season activates um, slow query lock and see whether you can. Um, truncate tables or also try to optimize the statements within the application those are from my point of view. Really um, lots of lots of still low hanging fruits.
- Borys: Okay.
- Benno: Have a very big impact um on the overall performance of your store and also ensuring that you you cache the frontend tried cash translations cashche like ah the the parts of the sea. Those are also things that you really should take a look at. When when optimizing for performance I think that those are already ah ah 5 um, 5 ideas or 5 takeaways that are easy to do and which all have an an impact on the performance and they are even more like.
- Borys: Um.
- Benno: Activating http 2 or um and a bigger or maybe a not too big thing. You could go with webp images and um, use the um friends of shopper extension. Um, for this.
- Borys: True true.
- Benno: That's that's all things minor things. Um, which can really um, differentiate you regarding the performance from your competition and in order to to get the most out of um, the festive season is to have. Ah.
- Borys: Um.
- Benno: Yeah, ah, really fast store. Um.
- Borys: Know? Yeah, but as you as we talk you, you cannot do it on your own as ah as a shop owner you need to have the right right? partner partner for this. Ah, that's it. So when usually.
- Benno: Okay, that but yeah, ah maybe um, when when I say you should do that in general you lock into our managed center.
- Borys: I.
- Benno: You do the shop performance report and you get in touch with us and our admins will also take care of those things. Those are the the ones if we as long as it's configurations as long as it's like ah configuring ah varnish or configuring Redis or configuring Elasticsearch. Those are all things that we do as a managed hostster and this is also something um, which helps agencies. Um, to yeah to to compete because we are really skilled in this kind of things.
- Borys: So.
- Benno: And um, and and do it together with your agency or um, solely on um on the merchants inquiry.
- Borys: Like yeah and when usually people should do what? yeah because during I mean right now it's already too late. Yeah, you know what? I mean it's It's ah.
- Benno: That would be a fix if we if we see this is the bottleneck. Ah, that's ah, a hot fix. Um, and we try to not um, operate in ah such a way. Yeah, because like when we see this is a problem.
- Borys: Um.
- Benno: Your clients will see it as well as a problem and you will already pay for traffic. That's not converting and that's the the the bad thing you could come up with so in order to um in order to prepared ah to be prepared for the season we start like. Eight weeks ahead of the event.
- Borys: So this is the preparation So when is the best time to do the migration or where you when people should start already doing this.
- Benno: I think two months and advance is the best thing because that will give you enough time to also plan your promotions. Um or you are able to do even load testing on ah on a proper setup to see. Whether your expected um traffic can be stand and this is like the the best timing I would say two months in advance is a very good timing.
- Borys: So if someone who's listening to this to this episode then probably ah no one will listen to us in next few days because of the because of the black friday but they will listen just after. Anything they can do or consult right now or you think it's ah it's a matter of like January of February to to consider. Okay, we failed with ah black friday so now it's a time to to find a better partner.
- Benno: A very good Question. So if the partner the hosting partner and the agency is not able to do it. They are not no. Um I would say. No low-hanging fruits how you can really achieve a big a bigger amount in traffic. You can try to scale ah by scaling up resources. It always helps to a certain amount for sure adding more Ram adding more cpu.
- Borys: No.
- Benno: Ah, that will help but um, only to um, yeah, a lower extent.
- Benno: Um I think um Cloud can solve many problems but especially during these peak events. We've seen in the past that Cloud offerings don't stand um to have combined load.
- Borys: But.
- Benno: Like ah if a cloud offering um has to deal with not like it's not a dedicated cluster that you use as 1 client so everybody has um, highlight scenarios and we've seen that. Apis and response times in the past um slowly um, got worse um to even to such an extent that they broke and then um, this is like the worst case and the worst scenario you.
- Benno: Wouldn't want to see because then automatically means you don't have somebody to call that will fix it for you. It will just be gone but those are extreme scenarios that happened in the past. Um.
- Borys: And.
- Benno: The the things I really see um cloud can't do as good as on-prem right? now is custom solutions and also delivering all the line of business applications. On one host with one. Um, ah one um, company or hoster to call so when I'm speaking off line of business applications. Those are the um. Auto management systems. Those are the ah pim pm systems um cms etc that have have to be orchestrated in the bigger um scenarios and if you have like 2 or 3 cloud offerings and.
- Borys: And.
- Benno: Have to orchestrate all of them. You have to also glue them together with another software which is then a single source of failure and you have to monitor it and that has to be hosted somewhere. So. Therefore I don't think Cloud is the solution for everything that solves everything automatically, but there are for sure many use cases where Cloud is a very very appealing and right solution.
- Borys: Cool, very good very good answer ah yeah so I think yeah at that point it's it's good to like ah I'll keep all my finger crossed for all the hosting companies and all the shops to go well through. Ah, black friday yeah, especially to do all the orders because yeah, it's it's right now. So important to keep the sales sales ah up and running and beno. Yeah, thank you very much I think you are quite good because we are talking for like 40 minutes and there was no calls about and a and they and escalations anything so that's ah.
- Benno: Um.
- Borys: That's ah, that's I hope a good. It's ah it's a good. It's a good prognostic so it was great and was nice to have you here. Thanks.
- Benno: Yeah, thanks again for the invitation and let's see what black Friday holds for us.