This is part two of a longer post. Read part one.
While we made huge strides with our build pipeline, our hosting was lacklustre. We served assets out of a single AWS region but our business was starting to expand globally. We tested load times from Brazil and it was unbearable. Being a single-page application did help. If the user was patient enough to wait for the giant bundle.js to be downloaded, they were rewarded with a fairly good experience afterwards. We did our best to animate actions before the network request was completed and the users were pretty happy with that. But we had to do something about the initial load. We need to move onto a Content Distribution Network.
Our initial search resulted in us being one of the earliest customers of Netlify. We immediately hit a snag, Battlefy runs on a naked domain, i.e., it’s https://battlefy.com not https://www.battlefy.com. There is a limitation with naked domains, you cannot have a CNAME records (points to domain), you can only have alias records (points to IP address).
This was a problem with our integration with Netlify as they used CNAME records in order to route traffic to the closest edge node for the user. They did this by returning the closest edge node IP when solving the domain set to the CNAME record. As we can only use alias records for the naked domain, it means we would be forced to point to the IP of one particular edge node. This would mean we would have no performance gain over our existing setup.
Fortunately, we were able to work closely with the founders of Netlify and test a new feature they were adding, an anycast network. This allows the traffic to be routed to the edge node using the same alias record IP without extra hops. We were able to point our naked domain alias record to Netlify’s anycast network IP and it worked!
We retested the load times from Brazil and we saw two orders of magnitude improvement! Our users instantly noticed and celebrated with us.
Netlify also did the build for us. It directly watches for pushes to our master branch on GitHub. It aligned perfectly with how we were doing things already, how convenient.
A new ability was unlocked as well. Netlify supported instant rollbacks. This wasn’t something we had before. We were amazed we survived this long without it.
We were happy for a long time with this setup.
What we’ve eliminated thus far
Over time our site kept getting larger. Our bundle.js kept getting larger. Our build time kept getting longer.
Our main frontend codebase was starting to age as well. Battlefy was originally built on Angular 1.5. We didn’t like the direction Angular 2 was going in, so we waited. We got lucky, we waited long enough for React to show promise.
We wanted to start doing React, but how? After looking at a few options, I came up with what I call “microapps”. The gist of it is, have Webpack build multiple entry points, in our case Angular microapp and React microapp. We can use ui-router and react-router within the microapps, but then use <a href=...> to jump between microapps.
Each microapp is served under its own context path. In our case, we had our existing Angular app served under /* and new pages like /settings/* serve the React microapp. This needs to work locally with webpack-dev-server and also needs to be supported by Netlify. The solution to both are rewrite rules. I’ll talk more on this later, as rewrite rules became a huge issue in the future.
This solves the growing bundle.js problem, as now each microapp has an independent bundle.js. This also improved the load times for our users as browsers only need to load the bundle.js for the microapp they are on, which could be very small indeed!
We wrote React and for a time, we were happy. New pages were faster to develop, we had plenty of things to learn and figure out. But then we started to discover issues with microapps.
The biggest issue with microapps was both the Angular app and React apps (now multiplying quickly) were in the same npm project. This means shared dependencies. If we needed to upgrade a dependency to fix a bug, we had to test multiple microapps to ensure it didn’t have unintended consequences.
The webpack start time was also pretty bad. If we added a new tiny react microapp, webpack to still need to build everything before we could start working on the new microapp. This wasn’t sustainable.
Another thing that wasn’t sustainable was our Netlify bill. At this point, we’ve been with Netlify for a few years. As we were an early adopter, we benefited by having favourable pricing and access to direct support from the founders. As you may know today, Netlify became a huge success, but along with success came changes in their pricing model and formalization of their support process. Netlify no longer made sense for us. The buy vs build equation tipped over.
What we’ve eliminated thus far
Read part three of this series as we continue our journey to CloudFront.
If you want to learn from me on how to build simple effective systems, you’re in luck, Battlefy is hiring! Check out our open positions
If you have any questions or comments, feel free to tweet me at pyrolistical
