TDM 30100: Project 11 — 2022

Motivation: One of the primary ways to get and interact with data today is via APIs. APIs provide a way to access data and functionality from other applications. There are 3 very popular types of APIs that you will likely encounter in your work: RESTful APIs, GraphQL APIs, and gRPC APIs. Our focus for the remainder of the semester will be on RESTful APIs.

Context: We are working on a series of projects that focus on building and using RESTful APIs. We will learn some basics about interacting and using APIs, and even build our own API.

Scope: Python, APIs, requests, fastapi

Learning Objectives

  • Understand and use the HTTP methods with the requests library.

  • Write REST APIs using the fastapi library to deliver data and functionality to a client.

  • Identify the various components of a URL.

Make sure to read about, and use the template found here, and the important information about projects submissions here.

Questions

We updated this project on Tuesday. This update includes changes to the commands run on a windows machine in question (3). We updated the commands as well as the shell (powershell instead of cmd). As noted in the updated question, please make sure to run powershell as an administrator before running the provided commands.

Question 1

If at any time you get stuck, please make a Piazza post and we will help you out!

For this project, we will be doing something a little different in order to try to make the API development experience on Anvil more pleasant. In addition, I imagine many of you will enjoy what we are going to setup and use it for other projects (or maybe even corporate partners projects).

Typically, when developing an API, you will have a set of code that you will update and modify. To see the results, you will run your API on a certain port (for example 7777), and then interact with the API using a client. The most typical client is probably a web browser. So if we had an API running on port 7777, we could interact with it by navigating to localhost:7777 in our browser.

This is not so simple to do on Anvil, or at least not very enjoyable. While there are a variety of ways, the easiest is to use the "Desktop" app on ondemand.anvil.rcac.purdue.edu and use the provided editor and browser on the slow and clunky web interface. This is not ideal, and is what we want to avoid.

Don’t just take our word for it, try it out. Navigate to ondemand.anvil.rcac.purdue.edu and click on "Desktop" under "Interactive Apps". Choose the following:

  • Allocation: "cis220051"

  • Queue: "shared"

  • Wall Time in Hours: 1

  • Cores: 1

Then, click on the "Launch" button. Wait a minute and click on the "Launch Desktop" button when it appears.

Now, lets copy over our example API and run it.

  1. Click on Applications > Terminal Emulator

  2. Run the following commands:

    module use /anvil/projects/tdm/opt/core
module load tdm
module load python/f2022-s2023
cp -a /anvil/projects/tdm/etc/hithere $HOME
cd $HOME/hithere

  3. Then, find an unused port by running the following:

    find_port # 50087

  4. In our example the output was 50087. Now run the API using that port (the port you found).

    python3 -m uvicorn imdb.api:app --reload --port 50087

Finally, the last step is to open a browser and check out the API.

  1. Click on Applications > Web Browser

  2. First navigate to localhost:50087

  3. Next navigate to localhost:50087/hithere/yourname

From here, your development process would be to modify the Python files, let the API reload with the changes, and interact with the API using the browser. This is all pretty clunky due to the slowness of the desktop-in-browser experience. In the remainder of this project we will setup something more pleasant.

For this question, submit a screenshot of your work environment on ondemand.anvil.rcac.purdue.edu using the "Desktop" app. It would be best to include both the browser and terminal in the screenshot.

Items to submit

  • Code used to solve this problem.

  • Output from running the code.

Question 2

The first step in this process is an easy one. Install VS Code on your local machine. This is a free, open source, and cross-platform editor. It is very popular and has a lot of great features that make it easy and enjoyable to use.

For this question, submit a screenshot of your local machine with a VS Code window open.

Items to submit

  • Code used to solve this problem.

  • Output from running the code.

Question 3

You may be wondering how we are going to use VS Code on your local machine to develop on Anvil. The answer is we are going to use a tool called ssh along with a VSCode extension to make this process seamless.

Read through this page in order to gain a cursory knowledge of ssh and how to create public/private key pairs. Generate a public/private key pair on your local machine and add your public key to Anvil. For convenience, we’ve highlighted the steps below for both Mac and Windows.

Mac

  1. Open a terminal window on your local machine.

  2. Run the following command to generate a public/private key pair:

    ssh-keygen -a 100 -t ed25519 -f ~/.ssh/id_ed25519

  3. Click enter twice to not enter a passphrase (for convenience, if you want to follow the other instructions, feel free).

  4. Display the public key contents:

    cat ~/.ssh/id_ed25519.pub

  5. Highlight the contents of the public key and copy it to your clipboard.

  6. Navigate to ondemand.anvil.rcac.purdue.edu and click on "Clusters" > "Anvil Shell Access".

  7. Once presented with a terminal, run the following.

    mkdir ~/.ssh
vim ~/.ssh/authorized_keys

# press "i" (for insert) then paste the contents of your public key on a newline
# then press Ctrl+c, and type ":wq" to save and quit

# set the permissions
chmod 700 ~/.ssh
chmod 644 ~/.ssh/authorized_keys
chmod 644 ~/.ssh/known_hosts
chmod 644 ~/.ssh/config
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub

  8. Now, confirm that it works by opening a terminal on your local machine and typing the following.

    ssh username@anvil.rcac.purdue.edu

  9. Be sure to replace "username" with your Anvil username, for example "x-kamstut".

  10. Upon success, you should be immediately connected to Anvil without typing a password — cool!

Windows

This article may be useful.

  1. Open a powershell by right clicking on the powershell app and choosing "Run as administrator".

  2. Run the following command to generate a public/private key pair:

    ssh-keygen -a 100 -t ed25519

  3. Click enter twice to not enter a passphrase (for convenience, if you want to follow the other instructions, feel free).

  4. We need to make sure the permissions are correct for your .ssh directory and the files therein, otherwise ssh will not work properly. Run the following commands from a powershell (again, make sure powershell is running as administrator by right clicking and choosing "Run as administrator"):

    # from inside a powershell
# taken from: https://superuser.com/a/1329702
New-Variable -Name Key -Value "$env:UserProfile\.ssh\id_ed25519"
Icacls $Key /c /t /Inheritance:d
Icacls $Key /c /t /Grant ${env:UserName}:F
TakeOwn /F $Key
Icacls $Key /c /t /Grant:r ${env:UserName}:F
Icacls $Key /c /t /Remove:g Administrator "Authenticated Users" BUILTIN\Administrators BUILTIN Everyone System Users
# verify
Icacls $Key
Remove-Variable -Name Key

  5. Display the public key contents:

    type %USERPROFILE%\.ssh\id_ed25519.pub

  6. Highlight the contents of the public key and copy it to your clipboard.

  7. Navigate to ondemand.anvil.rcac.purdue.edu and click on "Clusters" > "Anvil Shell Access".

  8. Once presented with a terminal, run the following.

    mkdir ~/.ssh
vim ~/.ssh/authorized_keys

# press "i" (for insert) then paste the contents of your public key on a newline
# then press Ctrl+c, and type ":wq" to save and quit

# set the permissions
chmod 700 ~/.ssh
chmod 644 ~/.ssh/authorized_keys
chmod 644 ~/.ssh/known_hosts
chmod 644 ~/.ssh/config
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub

  9. Now, confirm that it works by opening a command prompt on your local machine and typing the following.

    ssh username@anvil.rcac.purdue.edu

  10. Be sure to replace "username" with your Anvil username, for example "x-kamstut".

  11. Upon success, you should be immediately connected to Anvil without typing a password — cool!

For this question, just include a sentence in a markdown cell stating whether or not you were able to get this working. If it is not working, the next question won’t work either, so please post in Piazza for someone to help!

Items to submit

  • Code used to solve this problem.

  • Output from running the code.

Question 4

Finally, let’s install the "Remote Explorer" or "Remote SSH" extension in VS Code. This extension will allow us to connect to Anvil from VS Code and develop on Anvil from our local machine. Once installed, click on the icon on the left-hand side of VS Code that looks like a computer screen.

In the new menu on the left, click the little settings cog. Select the first option, which should be either /Users/username/.ssh/config (if on a mac) or C:\Users\username\.ssh\config (if on windows). This will open a file in VS Code. Add the following to the file:

mac config
Host anvil
    HostName anvil.rcac.purdue.edu
    User username
    IdentityFile ~/.ssh/id_ed25519
windows config
Host anvil
    HostName anvil.rcac.purdue.edu
    User username
    IdentityFile C:\Users\username\.ssh\id_ed25519

Save the file and close out of it. Now, if all is well, you will see an "anvil" option under the "SSH TARGETS" menu. Right click on "anvil" and click "Connect to Host in Current Window". Wow! You will now be connected to Anvil! Try opening a file — notice how the files are the files you have on Anvil — that is super cool!

Open a terminal in VS Code by pressing Cmd+Shift+P (or Ctrl+Shift+P on Windows) and typing "terminal". You should see a "Terminal: Create new terminal" option appear. Select it and you should notice a terminal opening at the bottom of your vscode window. That terminal is on Anvil too! Way cool! Run the api by running the following in the new terminal:

module use /anvil/projects/tdm/opt/core
module load tdm
module load python/f2022-s2023
cd $HOME/hithere
python3 -m uvicorn imdb.api:app --reload --port 50087

If you are prompted something about port forwarding allow it. In addition open up a browser on your own computer and test out the following links: localhost:50087 and localhost:50087/hithere/bob. Wow! VS Code even takes care of forwarding ports so you can access the API from the comfort of your own computer and browser! This will be extremely useful for the rest of the semester!

For this question, submit a couple of screenshots demonstrating opening code on Anvil from VS Code on your local computer, and accessing the API from your local browser.

Items to submit

  • Code used to solve this problem.

  • Output from running the code.

Question 5

There are tons of cool extensions and themes in VS Code. Go ahead and apply a new theme you like and download some extensions.

For this question, submit a screenshot of your tricked out VS Code setup with some Python code open. Have some fun!

Items to submit

  • Code used to solve this problem.

  • Output from running the code.

Please make sure to double check that your submission is complete, and contains all of your code and output before submitting. If you are on a spotty internet connection, it is recommended to download your submission after submitting it to make sure what you think you submitted, was what you actually submitted.

In addition, please review our submission guidelines before submitting your project.