How to conveniently transfer data to and from ftp servers using dbsync - Part 3

In this series of three articles we explain why it is advantageous to use DBSync to transfer data from an FTP server to Salesforce, and how to do it.

In the first part, we have seen how to create a connection between DBSynk and Salesforce, and between DBSync and an FTP server. In the second article, we saw how to create a process, a workflow, a trigger, a rule and a mapping, and how to configure the post processing. In this part 3, we will see how to run the process, and how to use the log.

Step 10: Run the process

A process can be run in two different ways: manually and using the scheduler. A manual run is useful in tests or sporadic use. The scheduler lets us manage repeated runs, or runs at a specific date and time.

A – Manually

In order to do a manual run, we simple need to press the Run Now button.

Figure 21 – Manual run.

B – Scheduler

The scheduler lets us to run a process at previously defined dates and times. It also lets us to repeat the execution periodically.

We can create more than one schedule. All defined schedules are shown on the screen, and listed in tabular manner.

Figure 22 – Create a schedule.

Additional Resource: Check the logs.

In case of problems, it can be useful to see the logs. There, we can find information on completed processes, errors, run times, and more. The log screen contains some useful filters, and an option to download the report. Figure 23 presents a typical log view.

Figure 23 – Typical log view.

Final Words

Throughout this series of three articles we have seen that DBSync presents a very convenient manner to transfer data from an FTP server to an application, and vice versa.

We have also seen an example using Salesforce that explains how a data transfer can be done in ten easy steps.

Similarly, DBSync lets us to transfer data to/from Salesforce from/to other apps. Thus, this tool gives us the capacity to integrate our different sources of data, creating synergies and new business opportunities.

How to conveniently transfer data to and from ftp servers using dbsync - Part 2

In this series of three articles we explain why it is advantageous to use DBSync to transfer data from an FTP server to Salesforce, and how to do it.

In the first part, we have seen how to create a connection between DBSynk and Salesforce, and between DBSync and an FTP server. In this second article, we will see how to create a process, a workflow, a trigger, a rule and a mapping, and how to configure the post processing.

Step 4: Create a process

Once we are connected to the source and the destination, we need to create a process that defines the task we want to do. For this, we press the button Create New Process, and we input a name. In our example we use the name “MoveMyData”. We must remember that process names cannot contain spaces.

Figure 9 – Create a new process.

And we get a confirmation of the newly created process:

Figure 10 – Confirmation.

Step 5: Create a workflow.

A process contains one or more workflows. In order to create a new workflow, we must press the Create New Workflow button. The system will ask for a workflow name, which in our example is “SF_to_FTP” (figure 10). Once again, we must remember not to use empty spaces in our workflow name.

Figure 11 – Create new workflow.

After pressing the button Save, we will be prompted to a screen that shows the newly created workflow.

Figure 12 – Workflow information.

It is important to know that we can create more than one process. Those processes will run in sequence. In addition, each process has a status, which can be ON or OFF. If the status is OFF, the process will be ignored during the run.

For example, in figure 13, we have defined three process. Process number two has its status set to OFF. As a consequence, when we run the sequence, the order will be: process 1 followed by process 3. Process 2 will be ignored.

Figure 13 – Workflow sequence.

Step 6: Configure the trigger.

We need now to configure a trigger, which will contain a query to Salesforce. Figure 14 shows how to do this.

Figure 14 – Trigger configuration.

Once we have constructed our query, we can check the results by clicking on the Preview button. The results of the query will be displayed under the Query Result section.

Figure 15 – Query preview

We can also use the Advanced Query Builder. This option presents us with the possibility of directly working with the SQL query statement, manually making the necessary modifications. In order to do this, we must click on Switch to Advanced View.

Figure 16 – Advanced Query Builder

Furthermore, we can click on the Preview button and check if the result of the query is what we’re expecting. Once we are happy with our modification, we can Save and Close, or if we don’t want to change the original query, we can simply Close.

Figure 17 – Advanced View

Step 7: Create a rule.

In the Rules section we define the properties of the file(s) that will store the results of our query. In our case we leave Mode default’s value (passive), we need an ASCII file type to store our records, and we define the name of the file that will contain the records as “infile.csv”. The Batch field defines the number of records that will be processed at once. Finally, we write the name of the directory that will contain our file(s).

Note: A practical way to name the file(s) is by including the date. In this case, the name syntax is “infile_{0,date,dd-MM-yyyy-HHmmss}.csv”.

Figure 18 – Target Properties

Step 8: Define a mapping

A mapping describes the relationship between the fields in the FTP server and the fields in the application. In our case, need to link the fields in Salesforce to the fields in our target file. Figure 20 shows how to do this.

Figure 19 – Mapping

Step 9: Configure the Post Processing.

Post processing allows us to configure actions to be taken after the process has been completed. The action can be any operating system command, or an executable file. The command may include the name of a file, which is defined by a regular expression. If the File Has Header field is set to True, then the operation includes header details in the result. The fields Directory and Trigger Name must be left blank. In our example we won’t include any post-processing instruction.

Figure 20 – Post processing

Wrapping up Part 2

In this part 2 of this series of three articles, we have shown how to create a process, a workflow, a trigger, a rule, and a mapping, and to configure the post processing.

In the next article, we will run this project and see the results.

How to conveniently transfer data to and from ftp servers using dbsync

The File transfer protocol is a standard network protocol used to transfer computer files from one to another host over a TCP network, such as the internet. The standard is widely used due to its convenient characteristics.

Sometimes we need to transfer the data stored in an FTP server to an application, such as Salesforce, MySQL, or vice versa.

There are several tools that can help us to perform this task. However, not all of them have the same capacities. Among them, DBSync stands out by its task automation ability.

Automating your data transfer

Different from other apps, the DBSync lets us transfer file data from an ftp server to any other source available in DBSync, such as Salesforce, and to move and delete files stored in an ftp server in an automated manner.

In order to do this, DBSync uses the FTP Connector, which implements a file transport channel that lets you transfer files between DBSync and an external FTP server. This can be done in two modes:  Trigger (from where a file is read) or Rule (writes a file to an FTP server).

The DBSync Connector allows you to do this with multiple files, and has the ability to select them by using regular expressions. In addition, it provides you with the option to do post-processing operations. This rich capacity gives the DBSync connector an unparalleled advantage, as it frees up your time and reduces your probability of making mistakes.

How to use the FTP connector

Example: transferring data from a file stored in an FTP server to a Salesforce custom object. In this example we will transfer data from Salesforce to an FTP server and vice versa. Let’s solve this problem with DBSync in ten easy steps.

Step 1: Create a new project.

When it comes to defining a project, we have two options: importing an existing one, or defining a new project. In this example, we will define a new project.

For this, we need to press the button Create New Project. As a result, we will get a dialog box asking for the name of our project. We input MoveData, and then press the Save button.

Figure 1 – Create a new project

Step 2: Create an FTP connector.

An FTP connector links DBSync to our FTP server. In order to create it, we need to press the Create New Connector button, and to define two parameters: the connector’s name and the type of connector.

Thus, we first write our connector’s name. In our example: MyFTPConnector. Note that the connector’s name cannot include any spaces.

Then, we define the connector’s type. Here the program presents us with a drop down menu, from where we select the option FTP connector.

Figure 2 – Create a new FTP connector

Once we have created our connector, we have to define its parameters. The most important aspect to understand here, is that there are two verification options: password and private key.

If we have choose a password, we just need to enter its value. (figure 3)

Figure 3 – Set up the connector (Password)

If we prefer to use a private key, we need to check the Use SSH Key for Authentication option and input the private key (figure 4).

Figure 4 – Set up the connector (Private Authentication Key)

Optionally, we can also add a public key and a pass phrase (figure 5).

Figure 5 – Set up the connector (Private Authentication Key)

Step 3: Create a Salesforce connection.

We need now to create a link to Salesforce. For this, we create a new connector of type Salesforce Connector, which we name MySalesforce. Once again, we must remember that the name cannot have any empty spaces.

Figure 6 – Create a Salesforce connector.

After pressing the Save button, the app shows a Salesforce login screen. Here we need to input our Salesforce login credentials.

Figure 7 – Salesforce login.

Once the credentials have been accepted, we see a screen confirming the successful connection.

Figure 8 – Salesforce connection confirmation.

Wrapping up Part 1

We have seen that DBSync has the advantage of automating the data transfer between Salesforce and an FTP server. In this first part of a series of three articles, we have also seen an example on how to connect DBSync to Salesforce and to an FTP server.

In part two, we will see how to create a process, a workflow, a trigger, a rule, and a mapping, and to configure the post processing.