How we used the cloud to supercharge remote development

The Rippling application is a Python monolith. Our repository has 1.91M lines of code and 8.6K Python files with 200+ developers contributing and merging 100+ PRs every day. 

When I joined the development team, the application was already massive enough to put significant strain on our standard-issue development machines. It required considerable memory and CPU effort to simply run. Each time we’d open PyCharm (our preferred IDE for development) and bring our entire stack up, the laptop’s resources were pushed to the limits. Add a Zoom session to the mix and the dreaded kernel_task would kick in and our laptops would become unusable. 

We needed to find a solution to our hardware resource challenges, one that played nice with a mostly remote development team and, most importantly, one that could handle a dynamic enterprise-scale Python repository. Here’s what we came up with. 

Working with remote servers 

From the start, we had a hunch that the solution to our hardware resourcing issues lay in the cloud. In June 2021, we started experimenting with offloading our application, and much of the compute load, to remote servers. 

Our primary consideration was to retain developers' existing workflow of using PyCharm to develop code, run tests, access the backend server, debug issues, and use known keyboard shortcuts. Due to this, we rejected all browser-based cloud development solutions such as AWS Cloud9 and Codespaces.

We wanted the UX of local IDE with the resource power and scalability of the cloud. To meet these requirements, we chose to use EC2 servers hosted within our AWS account for the compute power while retaining PyCharm as the IDE.

In this approach:

• Developers continue to write code in PyCharm

• As they write code, we sync it with the remote EC2 servers

• Developers use their usual shortcuts to run tests and debug the application

• Execution occurs on the EC2 servers with seamless integration to PyCharm’s output console

The sync & testing challenge

Our remote-first approach had many advantages over local machines, but we needed to solve two crucial requirements before implementing it company wide. 

• The sync must be seamless and transparent with low latency. Developers should not wait for their changes to propagate every time they write code.

• Retain the use of PyCharm as a built-in way of running/debugging tests. Developers can continue to use the same keyboard shortcuts, with minimal workflow disruption. 

Bringing it all together

To unite our new cloud development environment, we wrote a few scripts that would:

• Log in to AWS

• Identify the instance for the developer 

• Start the instance

• Start the sync using Mutagen along with port forwarding

To trigger the scripts, a few PyCharm configuration files came in handy. Users run a PyCharm configuration to start the sync or the backend server or start/shut down the instance.