Overview

Hyperpotamus n. [hyper·pot·a·mus] - YAML based scripting to automating interactions with REST APIs, HTML, and text processing.

"Hyperpotamus is like duct-tape for automating web stuff."

Hyperpotamus combines a few key principles to make an easy scripting mechanism.

• Scripts are written in YAML/JSON and so can be generated or manipulated programmatically.
• Pluggable actions represent processing commands that can be executed
• Typed data in the session can be initialized from CSV/JSON/YAML files and manipulated on the fly.
• Inline <% macro %> syntax makes it easy to use and transform session data with powerful, pluggable pipes operations
• Built-in actions for
  • Sending HTTP requests
  • Verifying/capturing data in HTTP responses
  • Formatting output as Text, JSON, or CSV.
  • Querying and manipulating session data
  • Logical branching and flow-control
  • ...many more and extensible via plugins.

Hyperpotamus allows you to write simple, human-readable scripts using any text-editor to describe a sequence of actions (like HTTP/HTTPS web requests). Hyperpotamus scripts support multi-step processes where information can be retrieved from the results of one action to print out or use in subsequent actions. For example, you may need to retrieve a listing of photos from a website page before you select one or all of them to download on a second page. Hyperpotamus uses powerful dynamic macros to customize parameters in the web-requests. The values for these macros can be passed in on the command-line, read from a spreadsheet, loaded from a JSON/YAML data-file, captured from a previous request, or read in from the user running the script at a prompt.

Pizza-order example

Take a look at this simple example:

request: # Send a request to httpbin.org with our pizza order
  url: http://httpbin.org/post
  method: POST
    json:
      crust: deep-dish
      toppings: [ pepperoni, extra-cheese ]
response:
  - json: # Capture the IP address from the response
      ip_address: "$.origin" 
  - print: Order sent from IP Address: <% ip_address %>

Without much explanation, you probably already have a good idea of what this script is doing:

1. Sends a POST request with a JSON payload to a public website (httpbin.org)
2. Captures a specific value from the JSON response into a variable called ip_address
3. Prints out a message to display the captured ip_address.

Sweet-spot use cases

• Reading in records from a .csv file and processing actions for each record.
• Invoking a RESTful API, substituting dynamic data, capturing data from the response and using the captured data for subsequent processing steps (like calling another API endpoint).
• Screen-scraping HTML pages and submitting HTML forms.
• Periodically monitoring a website or application for certain behavior.
• Load-testing a web-application or site
• Automated integration tests for a web-application.

Why might someone want to do this?

• Boss (at 7:30pm): Hey! We need to get all the products on this spreadsheet entered on our customer's website by 8:00am tomorrow morning, but it is taking our team of 5 people forever to fill out the 3-page form on their website. We are trying to copy/paste the fields for each row to submit these 8,000 items! We really need you to jump in and help us with the click/copy/paste/submit party and our team is desperate! You didn't have any plans for tonight did you? I'll buy pizza!!

• You: Sure -- I'm happy to help! Send me the spreadsheet and give me about 20 minutes... oh, and deep-dish, please. Can you order it "to-go"?

Sounds awesome, but do I have to be a ninja to use it?

No. You can write some pretty simple scripts in just a few seconds. Hyperpotamus scripting was designed to be easy and accessible with lots of shortcuts and defaults. In spite of that simplicity, however, there is plenty of power available when you are ready to dig deeper. And just in case the built-in power does not satisfy your automation-hungry cravings, you can extend and bend hyperpotamus to do your will by writing custom action plugins.