Deploy using MiniKF on GCP

Using Google Cloud Marketplace to deploy MiniKF (mini Kubeflow) on Google Cloud Platform (GCP)

MiniKF latestversion

This guide describes how to launch a MiniKF virtual machine (VM) on GCP. MiniKF is a single VM solution on the Google Cloud Marketplace, and installs:

Kubernetes (using Minikube)
Kubeflow
Kale, a tool to convert general purpose Jupyter Notebooks to Kubeflow Pipelines workflows (GitHub)
Arrikto’s Rok, a data management software for data versioning and reproducibility

MiniKF supports NVIDIA GPUs.

Before you start

Before installing MiniKF, make sure that your GCP project meets the minimum requirements described below:

Select or create a project on the GCP Console.
Make sure that you have the editor role role for the project. You can add a member to a project and grant them a Cloud IAM role using the Cloud Console.
Make sure that billing is enabled for your project. For more info, see the Modify a Project’s Billing Settings guide.
If you are using the GCP Free Tier or the 12-month trial period with $300 credit, note that you can’t run the default GCP installation of MiniKF, because the free tier does not offer enough resources. You need to upgrade to a paid account.
Read the GCP guide to resource quotas to understand the quotas on resource usage that Compute Engine enforces, and to learn how to check your quota and how to request an increase in quota.

Installing MiniKF on GCP

To install MiniKF on GCP, follow the steps below:

Go to the MiniKF page on Google Cloud Marketplace.
Click on the LAUNCH button
In the Configure & Deploy window, choose:

Deployment name: a name for your MiniKF instance
Zone: the GCP zone that you will deploy MiniKF
Machine type: a machine type with at least 2 vCPUs and 16 GB of RAM. If in doubt, use the default machine type, n1–standard-8, which has 8 vCPUs and 30 GB of RAM.
Boot Disk: a boot disk. This is the disk that holds the images that MiniKF needs. Use the default value of 200 GB SSD Persistent Disk.
Data Disk: a data disk. This is the disk that holds your snapshots. Use an SSD Persistent Disk of at least 500 GB capacity.
GPUs: add one or more GPUs if you are running computationally intensive ML workloads.

Then click Deploy:

Now wait for the VM to boot.
When the VM is up, ensure that MiniKF is up and running. Before visiting the MiniKF dashboard, ensure that Minikube, Kubeflow, and Rok (Snapshot Store) are up and running. To do so, click on the SSH button to connect to the MiniKF VM and follow the on-screen instructions.
MiniKF dashboard, username, and password. Now that Minikube, Kubeflow, and Rok are up-and-running, close the SSH browser window, as you will not need it anymore. On your current view, please find the MiniKF dashboard URL, MiniKF username, and MiniKF password. You will need them to log in to Kubeflow and Rok. You should see a table like the one below:
Log in to Kubeflow. Visit the MiniKF dashboard URL and log in to Kubeflow using the MiniKF username and password.
Log in to Snapshot Store (Rok). After logging in to Kubeflow, navigate to the Snapshot Store:

And log in to Snapshot Store (Arrikto’s Rok) using the MiniKF username and password.