Episode 3 | Boomi HTTP to disk

This article covers the steps for using Dell Boomi to retrieve US Patent Office data from their Google bulk storage site and storing to a local disk.

This project uses Dell Boomi, Dell Quickstart Data Warehouse Appliance and Toad products to analyze publicly available data. For more information about the goal and scope of the project, as well as a breakdown of the episodes, go to: episode 1

First project: zip file to disk

First, set up a Dell Boomi account to start working on this project. You can view a short video that highlights the process of creating a trial account and getting your Dell Boomi environment up and running here Boomi Getting Started: Video1.

                                                 Figure 1.              Using Dell Boomi as the foundation for this project

A simple process is in place to help you familiarize yourself with Dell Boomi. The project is broken into several small steps. The first step is to download a zip file from the USPTO bulk data site and unzip it before saving to disk. The tasks for the first process are listed below.

• HTTP connector
  • Connection
  • Operation
• Disk connector
  • Connection
  • Operation
• Data process-unzip
• Set properties-file name

Creating components

There are several methods in Dell Boomi for creating components. Creating the process is the main unit for a Dell Boomi project.

To create a component process:

1. In the Build tab, in the upper left hand corner, click Create Component.
2. Select Process from the drop-down list.
3. Create a name for this process and choose a subfolder (if you plan to create a separate folder for your project) and then click Create
4. In the Process Options message dialog box, accept the default setting by clicking OK. 

                                                                   Figure 2.              Creating a process in Dell Boomi.

The Configuration window for the Start connector opens automatically once the process is created. For this first project, the start connecter is an HTTP connector. If you click Cancel, a blank process page appears with the icon below.

                                                                   Figure 3.              Clicking Cancel presents a blank page with this icon.

HTTP connector

There are two sub-components when creating HTTP connectors: connections and operations.

1. On the HTTP shape in the project (in this case the Start shape in Figure 3), select Configure.
2. In the Start Shape dialog box, select HTTP Client and Get from the drop-down menus.

                                                                       Figure 4.              Selecting HTTP Client and Get.

HTTP connection

Refer to Figure 4 for this procedure.

To create an HTTP connection:

1. To create a new connection, to the right of the Connectionfield, select the (+) symbol.
2. In the Name text box, type USPTO 2012w1.
3. Insert the URL:   http://storage.googleapis.com/patents/grant_full_text/2012/ipg120103.zip
4. Click SaveandClose.

HTTP operation

Use Figure 4 for this procedure.

To create an HTTP operation:

1. To the right of the Operationfield, select the (+) symbol.
   Because the connector type is a Get, the HTTP Method should automatically be set to Get in the drop-down options.
2. In the Name text box, type HTTP get.
3. In the Content Type text box, type: text/xml.
4. Click SaveandClose.
5. Select OK.

Disk connector

To create a disk connector:

1. Drag and drop the connector shape into the process field (the Connection Action window opens automatically).
2. In the drop-down lists, select Disk and Send.

                                                                   Figure 5.              Creating a disk connector.

Disk connection

Use Figure 5 for this procedure.

To create a disk connection:

1. To the right of the Connectionfield, select the (+) symbol.
2. In the Name text box, type local USPTO.
3. In the Directory text box, type c:\uspto.
4. Select SaveandClose.

Disk Operation

Use Figure 5 for this procedure.

To create a new operation:

1. To the right of the Operationfield, select the (+) symbol.
2. In the Name text box, type send unique.
3. In the File Naming Option, select Create unique name if file exists.
4. Select Save and Close.
5. Select OK.

Data process (Unzip)

                                                                               Figure 6.              The Data Process shape.

To unzip:

1. Drag and drop the Data Process shape into the process field.
2. In the Data Process properties windows, label it Unzip.
3. Under Processing Steps, select the (+) symbol.
4. Select the following:
   1. Process Type:                Unzip
   2. Connector:                    HTTP Client
   3. Action:                          Get
   4. Property:                       Url
5. Click OK.

Set Properties (File name)

                                                                                   Figure 7.              Set Properties shape.

To set properties:

1. Drag and drop the Set Properties shape into the process field.
2. In the Set Properties windows, label FileName test.xml, and under the Properties to Set, select the (+) symbol.
3. Select Document Property and Standard Connectors->Disk->File Name and click OK.
4. Select theDisk – File Namethat was created in the Properties to Setwindow and then under Parameters, select the (+) symbol.
5. In the Type text box, type Static.
6. In the Static Value text box, type test.xml.
7. Click OK, and then OK again.

Completing the process

You now have all the components, but before you can start running, you need to put it all together.

To complete the process:

                                                                    Figure 8.              Stop shape completes the process.

1. In the Shapes menu, click Logic and then drag-and-drop a Stop shape to complete the process.
2. For Stop options, click OK.
3. If you have not already done so, drag-and-drop all arrows to the subsequent components so that the process flow is established. When completed, the process should look like this:
4. Once everything looks good, name your process First project and click Save and Close.
5. In Component Explorer, click Refresh.
6. Once your project is fully expanded, your project should look like this:

                                                                Figure 9.              The process flow is now complete.

                                                                          Figure 10.             The fully expanded project.

Running your first project

To deploy and run the process, view the video Boomi Getting Started: Video2. The service that is actually running the tests is the Atom mentioned in the video Boomi Getting Started: Video1. This means, that for this specific process, if the folder c:\uspto does not exist on the system where the Aton is running, the process fails.