Step-by-Step Guide on how to host your Trading Strategy in the Cloud

Marius SprengerAugust 19, 2021

Trading

Hey there. My name is Marius and I am a community developer at lemon.markets, a Berlin-based startup that is building an infrastructure so that developers can build their own brokerage experience at the stock market.

A use case that our users are frequently building is an automated trading strategy that makes automatic buy or sell decisions based on a specific predefined logic. Obviously, an important part of such a strategy is to make sure that it is constantly running, as the whole point of automating your strategy is to not have to worry about manually placing orders anymore after you’ve set it up. The solution for that is (you might’ve guessed it): cloud hosting. While three “Internet giants” provide cloud solutions (Amazon with AWS | Google with GCP | Microsoft with Azure), there are also a number of smaller providers, such as Heroku, Deta or DigitalOcean.

In this blog post, we want to tackle this (in our humble opinion) important topic and show you step-by-step how you can host your very own trading strategy in the cloud using Heroku. Additionally, we’ll dive a bit deeper into Deta, give you an overview of other cloud hosting providers and list their Pros and Cons.

Ready to learn about hosting your trading strategy in the cloud? Wonderful. Then let’s dive right in.

When we at lemon.markets think about automated trading strategies, we feel that it is a process that consists of multiple steps and phases. We try to explain the different phases in this article, and additionally provide you with tools that help you during each one.

While we advise you to dedicate a lot of your time to building a trading strategy that yields consistent return, it is our conviction that you should not waste your precious resources on the hosting part. That’s where this article comes into play: we want to give you a quick and efficient guideline that enables you to host your trading script conveniently and without any hassle, so you can focus on really nailing and tweaking your trading strategy. In the end, we feel that our role at lemon.markets is to provide you with a holistic resources experience, so you can start building whatever you have in mind right away and without too many distractions.

What is the right provider for me and my Trading Strategy?

There are a lot of cloud hosting providers out there, and essentially they all enable you to do a similar (or the same) thing: shift a project that you have running locally to the cloud, so that:

• it does not consume any of your local resources anymore, and
• keeps running, even if you shut down your computer.

Depending on what your script does, one provider might be more advantageous than the other. However, in this blog post, we will focus on providing a detailed description for working with Heroku for a number of reasons:

1. It easily integrates with GitHub and allows you to deploy your code super fast and with very little effort.
2. It is well suited for smaller, individual projects with comparatively low computing power (therefore perfect for the use case of our automated trading strategy).
3. Heroku has a free tier, which is more than enough for running a single automated trading strategy.
4. It allows you to scale up conveniently at very little cost if after a while you notice that you need more computational power.
5. It gives you flexibility for how you prefer to deploy your script, as you can either work with the Heroku Dashboard or use the Heroku CLI to deploy your script from your terminal.

As you can see, Heroku is a great alternative to conveniently host a single trading script with very little effort. However, check the last section of this blog post for a discussion on alternative cloud providers if you notice that Heroku may not be sufficient for your project (or if you are simply interested in learning more about the topic).

Step by step Guide for Heroku

For this article, we assume that you know your way around GitHub. One of the cool features of Heroku is the seamless GitHub integration, which makes our life a lot easier. But let us start at the beginning.

Preparing our Python project

In order to host a Python script on Heroku, you first need to have one (duh!). Let us assume that file is called main.py and that it is part of your Python project containing your automated trading strategy. In the main.py file, we define our main function that is executed on if __name__ == ‘__main__’. That is the file we need. By the way: you can take a look at this GitHub repo to see a specific Python project that contains a main.py file.

Before you push your project to GitHub, you need to add one specific file: the Procfile. To do so, simply add it to your project with, e.g.,

1touch Procfile

on a Mac. The Procfile does not have any ending, simply create it as above. In general, the Procfile is a specific file for Heroku apps that specifies what commands are to be executed when the app is started. If you are interested in learning more about the Procfile, click here.

Our Procfile contains only one line:

1worker: python main.py

What are we doing here?

We are simply following the syntax conventions of a Procfile, which is:

1<process type>: <command>

So, we want to assign a “worker” (a worker process is basically a process that executes a task in a Heroku application.) Additionally, we specify the worker task: executing a python script with the name main.py. And that’s it. Pretty simple, right? If you created your Procfile, you are ready to push your project to GitHub.

Preparing our Heroku Application

Next, we want to create our Heroku application. Log in to your Dashboard and click on “New” and select “Create new app” in the top right corner. You are presented with the following screen:

Create new Application in Heroku

Enter your app name there and choose a region. Afterwards, click “Create App”, which brings you to the application settings, where you need to do further refinements.

Here, you have a few options to deploy your application. For this blog post, we choose to connect our application to a GitHub repository (the one we created earlier that contains our Procfile). You might need to authorise Heroku to access your GitHub first. Afterwards, you can search for the repository you just created. Click on “Connect” if you identified the right one.

What if I don’t use GitHub?

While we think that integrating GitHub with Heroku is the most convenient way to host your trading script in the cloud and continuously updating it, we also acknowledge that (as with any situation in life), there might be varying preferences and approaches. Obviously, Heroku also provides other options. For example, you can deploy with Git directly using the Heroku CLI (find more information here), or with Docker (find more information here). If you prefer GitLab over GitHub, there is a nice article explaining how you can use GitLab with Heroku. However, we are going to stick with GitHub for this article, as we hope to address the needs of a majority of users :)

After choosing GitHub as your deployment method, you can select how you wish to deploy your application. You can either enable automatic deploys, which deploys the app to Heroku every time you push changes to your GitHub repository, or you opt for a manual deploy, which only triggers a deploy if you say so. For both options, you need to specify the branch you wish to deploy from. In most cases, it makes sense to use the main branch.

After you’ve clicked on either “Enable Automatic Deploys” or on “Deploy Branch”, Heroku starts (well 💁‍♂️) deploying your app. If everything went fine, you should see “Your app was successfully deployed” at the bottom. You can click on view to check the status.

If you click on “View”, you might be greeted with an application error.

Working with Environment Variables

In our case, this is because we work with environment variables in our project, which we need to set/define for the project to work. You can do so by either navigating to your project’s folder, logging into the Heroku CLI and setting the environment variables through:

1heroku config:set ENV_VAR=value

Alternatively, you can also set the environment variables in the dashboard directly. Navigate to Settings and go to the “Config Vars” section.

If you click on “Reveal Config Vars”, you can enter all your environment variables one by one:

After you’ve entered all environment variables, either in the Dashboard or through the Heroku CLI, you can re-deploy the project.

You can check the project status by typing

1heroku logs

into the CLI. If you notice that your app is not running properly, try typing:

1heroku ps:scale worker=1

And that pretty much concludes all the steps we need to take to get our Python script up and running. Simple, right? It is. We are really excited to hear about your automated trading strategies hosted in the cloud.

Scheduling

If you use the approach outlined above, what you essentially do is start a Heroku worker that executes your script and runs it. This means that the script is then constantly running in the cloud and actions and pauses are triggered within the script directly. This is one approach for running an automated trading strategy and we believe that it is a good starting point for you to shift your trading script from your local environment to the cloud (we ourselves have some test strategies running on Heroku using that approach and they yield some nice returns 😏. Stay tuned for specific articles on that in the upcoming weeks).

However, there might be scenarios where that approach is not the best possible solution and we want to quickly dive into a second approach: scheduling. Heroku offers the Heroku Scheduler Plugin and it may be a good solution for you and your trading strategy. Essentially, the scheduler allows you to define time intervals to run jobs on your app. It does not take that much brainpower to see the connection to our trading strategy use case here. If, for example, you have a trading strategy that is supposed to buy a stock at very specific times during the day, you could simply use the Heroku scheduler and define the times at which you want to place your order. Heroku then calls the script and runs the job just like you defined. If you are interested in that solution, check out the documentation to learn more.

Deta — an interesting alternative

By getting more familiar with the concept of scheduling, we stumbled across Deta a few months ago. Deta is an interesting alternative that is only just starting up in Berlin (which was another reason for us to check them out, since we are also building a tech company in a small Berlin office with a small but highly motivated team).

The idea behind Deta sounds promising: don’t worry about setting up your Python scripts in a complex way anymore, but simply deploy them in 2 seconds using the tiny but powerful Deta CLI. What we noticed straight away was that especially scheduling (or Cronjobs if we speak in Deta terms) seems really intuitive using Deta. Let’s take a closer look.

In the context of an automated trading strategy, the Deta Micros product is the relevant one. You can find specific and more detailed information about it here. On a high level, Deta works like this:

The first step you need to do is create a Deta account. You can sign up for Deta here. Activate your registration and sign in to their web app. The next important step is to install the Deta CLI through:

1curl -fsSL https://get.deta.dev/cli.sh | sh

Afterwards, you can log into the CLI with the command:

1deta login

This will open the Deta web app in your browser and authenticate your CLI. You can either create a new Deta micro, e.g., in Python through:

1deta new — python new_micro

1or add a micro to an existing project through:

1deta new — project <your-project>

Setting up Cronjobs and Deploying your Project on Deta

Make sure to add a requirements.txt file to your project, so that Deta knows what dependencies to install when deploying/running your project. In order to make sure that Deta is able to run your code, you need to create a file called main.py and create an app instance. To set up a cronjob (which, from a functionality perspective is basically the same as setting a scheduler in Heroku), you simply need to create a function that you want to run on a schedule and that takes an event as input:

1from deta import app
2@app.lib.cron()
3def cron_job(event):
4return “We are going to place an order now”

If you deployed that function on a Deta micro, you could afterwards conveniently execute the function by defining the cronjob through:

1$ deta cron set “10 minutes”

If you are interested in learning more about Cronjobs in Deta, take a look at thedocumentation. In general, you can deploy your scripts to Deta with:

1deta deploy

Afterwards, you can open up your micro to the public by disabling authentication:

1deta auth disable

And that was it. You just created your Deta micro and deployed it to the cloud. Make sure to check them out under deta.sh and play around with their services by letting an automated trading strategy run in the “deta cloud”. We might be featuring specific Deta tutorials in the future, so stay tuned 🤓.

Big players in the markets

When talking cloud hosting, there is almost no way around the big three: AWS, Google Cloud and Microsoft Azure. However, there is a reason why we will only discuss them marginally in this article.

All three providers are great infrastructures that allow you to build scalable projects in the cloud and maintain them flexibly. However, they might not be the best solution when you are trying to host a small project, as the setup on all three is more complex than, e.g. with Heroku or Deta. Heroku specifically is super simple to use compared to AWS, Google Cloud or Microsoft Azure, since you do not have to deal with the infrastructure management, as this is managed by Heroku automatically. Also, the GitHub integration is a big plus for Heroku compared to the other three providers, as this allows you to write flexible code and update your projects/scripts seamlessly. If you are interested in learning more about the discussion on Heroku vs. AWS/Google Cloud/Microsoft Azure, check out these resources:

This one compares Heroku and AWS, this one compares Heroku and Google Cloud, and this one (surprise) compares Heroku and Azure.

This concludes our blog post on hosting your automated trading strategy in the cloud. We hope you got a good insight into how you can connect your Trading Script to Heroku and therefore ensure that it is constantly running + a few insights on additional providers in the market and their advantages/disadvantages.

Do you have any comments or additional ideas/services that are missing in this blog post? Leave us a comment or contact us directly at support@lemon.markets. And if you haven’t done it, yet: sign up to get started with the lemon.markets API. We can’t wait to have you on board.

See you on 🍋 .markets soon.

Marius