Lab 5: Preventive Controls

In Lab 4, you tested out a remediating detective control that is triggered when a SageMaker training job is launched outside of the VPC. But waiting minutes to find out that your training job is going to error out is not a great experience for the data scientists. In this lab, you will implement a preventive control that will prevent a training job from starting if it’s not launched within a VPC. In the interest of defence in depth we will now implement the preventive control to complement the detective control exercised in the previous lab.

To deploy a preventive control, assume the role of the Data Science Administrator and create a new version of the SageMakerNotebookExeRole, updating it with one of the CloudFormation templates referenced below in Amazon S3.

Step-by-step instructions
  • Ireland (eu-west-1)

    https://sagemaker-workshop-cloudformation-eu-west-1.s3.amazonaws.com/execution_role_product_preventive.json
    
  • London (eu-west-2)

    https://sagemaker-workshop-cloudformation-eu-west-2.s3.amazonaws.com/execution_role_product_preventive.json
    
  • Sydney (ap-southeast-2)

    https://sagemaker-workshop-cloudformation-ap-southeast-2.s3.amazonaws.com/execution_role_product_preventive.json
    
  • Oregon (us-west-2)

    https://sagemaker-workshop-cloudformation-us-west-2.s3.amazonaws.com/execution_role_product_preventive.json
    
  • N. Virginia (us-east-1)

    https://sagemaker-workshop-cloudformation-us-east-1.s3.amazonaws.com/execution_role_product_preventive.json
    
  • Ohio (us-east-2)

    https://sagemaker-workshop-cloudformation-us-east-2.s3.amazonaws.com/execution_role_product_preventive.json
    

Now with a new version of the product defined turn to the Provisioned Products console and update the execution role created earlier to the latest version you just created.

Step-by-step instructions

After the product has been successfully updated resume the role of Data Scientist and revist the Jupyter notebook kernel and execute the cell titled Train again without a VPC. You should now quickly receive an Access Denied exception similar to the below:

ClientError: An error occurred (AccessDeniedException) when calling the CreateTrainingJob operation: User: arn:aws:sts::012348485732:assumed-role/SageMakerExecRole-ml-product-team/SageMaker is not authorized to perform: sagemaker:CreateTrainingJob on resource: arn:aws:sagemaker:eu-west-1:012348485732:training-job/sagemaker-tensorflow-2019-10-16-22-14-30-880 with an explicit deny

In this lab you modified the permissions granted to the instances of the data science team’s Jupyter notebooks. This altered their permissions so that they could only perform actions like creating a training job if that action met the security requirement of specifying a VPC configuration. Visit the defined products using the Service Catalog console to review the CloudFormation template and the changes it made to the permissions. Or use the IAM console to review the role you created and the policy attached to it. What conditions are on the IAM policy controlling access to the SageMaker API?