AWS: Workflow in SageMaker

An article to alleviate your confusion forever

Rahul S

--

AWS SageMaker is a fully-managed service for machine learning in the cloud. It lets us build and train ML models, and directly deploy them into a production-ready, hosted environment. Its various features include:

  • An integrated Jupyter notebook authoring instance. It offers easy access to data sources for exploration and analysis. There is no need to manage servers.
  • Common machine learning algorithms optimized to run against large data in a distributed environment. SM also lets you use your own frameworks and algorithms, offering flexible distributed training capabilities that can adjust to specific workflows.

I prefer SM console over Studio

WORK FLOW

In SM, ML modeling process comprises 1. preparation, 2. training, and 3. deployment. Following diagram illustrates typical workflow.

src: AWS

1. PREPARATION

SageMaker creates a fully-managed ML instance in Amazon EC2 and runs Jupyter computational processing notebooks.

SM can pull massive amount of data stored in Amazon Simple Storage Service (S3). The basic code prepared by us and packaged in a Docker container image can be imported by SM for further development and deployment.

2. TRAINING

If the computations is many, which is usually the case, we define the data’s location in an S3 bucket and specify a preferred instance type. We perform feature engineering. Post this, we begin the training phase.

You can train a model in SageMaker by creating a training job. Each training job needs to include the following information:

  • Training data storage —the URL of the Amazon S3 bucket that stores your training data.
  • Compute resource — the required compute resources for model training. A compute resource is a machine learning compute instance managed by SageMaker.
  • Output storage — the URL of the desired S3 bucket for storing the…

--

--