Neuro Platform: ML Developers Forum

[01.09.2019] FloydHub

This overview describes the state of the platform on September 1, 2019.

Overview procedure description

Product website

Branch in benchmark project

Developer experience - 2

This platform is more than just a Kubernetes or Docker Swarm, it is exactly the platform for machine learning. In other words, some assumptions and tweaks are made: PyTorch/TensorFlow out of the box, the requirements file, a special location for the training output, default TensorBoard, and so on.

Pros:

  • No need to interact with docker.
  • Convenient abstractions, such as a dataset, output or project.
  • Clear billing and resource monitoring.
  • Workspace out of the box (colab jupyter).
  • Simple and clear commands.
  • You can do a lot of things using web UI.

Cons:

  • For more difficult tasks it is probably not flexible enough.
  • It is not always clear how to do this or that action. For example, which path to access the data uploaded to the workshop.
  • No way to ssh connect to your running job.
  • No remote debugging.
  • Tensorboard is available with one click, but only while the training is running.

ML environments extensibility - 2

The platform provides numerous environments with PyTorch and TensorFlow. You can also install python and non-python additional requirements. But still, the platform user depends on which frameworks and their versions are provided by the platform developers.

Data Ingestion - 3

Working with data is possible via the dataset abstraction. You can upload data both from your local machine and from the internet; you also can resume your previous interrupted uploading. Additionally, the platform provides versioning of uploaded datasets. All training output is stored in a special place. As for the disadvantages, commands like ls or pwd are inconvenient because they create and run new job.

AI starter kits - 3

The platform developers provide a lot of examples and tutorials from different ML spheres. Also, they provide free 20 hours of cpu and 2 hours of gpu.

Collaboration - 2

Starting from the Team plan, you can share datasets, projects, and jobs with your teammates. However, the platform doesn’t have a native way to expose your model via jupyter notebook to the outer world.

Bring your own cloud - 3

The Platform website contains information about a possibility to run FloydHub in your own cloud, be it AWS, GCP, or Azure. However, there is no evidence of actual support of this possibility, just a call to contact the FloydHub support team.

Enterprise-ready - 2

In case of your choice of Team or Enterprise plans, you can benefit from various enterprise-ready features, like single sign-on, individual and team resource consumption reporting, and role-based security. Audit logs, though, are not available.

Final score - 17