There are quite many tutorials that explain to you how to run Tensorflow on an Apple Silicon machine with Miniconda, but I haven't seen any that show you how to do the same with Virtualenv which I've been using for my Python development.
So, in this article, I would like to show you how to install Tensorflow and run it inside a Virtualenv environment on an Apple Silicon machine while utilizing the GPU.
What is Virtualenv?
Before we start talking business, let's have a quick recap. What is Virtualenv?
If you start coding in Python you quickly learn two things: one is that there are different versions of Python and the other is that there is a package manager called PIP which helps you to install packages for your projects.
The problem with these two things is that they are global. Meaning that PIP, for every Python version that you've installed, manages the packages across your entire system. So, if you are using Python 3.11 and decide to install a specific package with a specific version, this package with that specific version will be used across all of your projects that use Python 3.11.
How to solve this?
Virtual environments are here to help!
Virtual environments in Python allow you to create an isolated environment for a specific project thus enabling you to set up a specific Python version for every environment and to install different packages (modules) with specific versions in each such environment without worrying about any collisions.
Several different providers for such environments can be used, such as Virtualenv, venv, Anaconda, Conda, and Miniconda.
Virutalenv is very powerful and can create virtual environments for different Python versions - the downside is that you have to install it yourself and you have to have sudo privileges on the machine on which you are installing it.
venv comes with Python (from version 3.3) and is based on Virtualenv but it cannot be upgraded, is a bit slower, and is bound to the specific Python version that you are using, but, you don't have to install it.
Anaconda is a Python distribution that combines Python, a package manager, and a virtual environment provider (Conda) and is a very heavy, but comprehensive, solution.
Conda, taken from their webpage, is a Package, dependency, and environment management for any language—Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, Fortran, and more.
Miniconda is just a free minimal installer for conda.
Because I'm just a Software Engineer and not a Data Scientist I use Python for a big variety of projects which include scripting, simple application servers, applications, DIY projects, ML/AI projects, etc., I have different Python versions that I use and I like my virtual environment configuration to reside closer to my project. These requirements eliminate Anaconda, Conda, and Miniconda because by default they use a centralized location for the environment configuration.
I also don't use venv because it is only available since version 3.3 and I still have older Python projects that I need to maintain.
So, Virtualenv is the best solution for me (tell me in the comments about your favorite virtual environment).
Let's Install
Now that we know what Virtualenv is and why we use (or maybe it's just I) it, we should go ahead and install everything.
We will be using brew to install it all, so first, let's install brew:
Now that we have brew we can install Python. We will be installing Python 3.11:
The last thing that we need to do is to install Virtualenv:
Create Our Project
Now that we have everything installed, let's create our Python project using Virtualenv and Python 3.11.
First, go to the folder in which you want to create your Python project folder and create the project folder:
Now we should create the virtual environment and run it. But before we can do that, we need to know where is our Python 3.11 installed so that we can tell Virtualenv about its location. To do that, we will use brew:
In my case, it said that the location of my installation is: /opt/homebrew/bin/python3.
Okay, now we can create the environment:
Let's activate it:
Now your prompt should look like this:
Installing Tensorflow
Congratulations, you've created your Virtualenv environment. Now you can install Tensorflow and use it in this project. Remember that you are running on Apple Silicon, so things are a bit different here...
We need to install two different packages: tensorflow-macos and tensorflow-metal.
That's it, we did it!
Let's check to see that it worked. Start by opening a Python console:
Now import Tensorflow, verify its version, and check if it has detected the GPU.
You should now be able to see the Tensorflow version and two physical devices one of which is the GPU.
Conclusion
In this article, we've learned how to use Virtualenv and install Apple Silicon-compatible Tensorflow packages that can utilize the embedded GPU. Try it out and tell me what you think in the comments.