Difference between revisions of "Internet Mail Archive System"

From PresenceWiki
Jump to: navigation, search
Line 190: Line 190:
 
#Run the task, Instructions from the Instructions Category.This will essentially, create the IMAS.CustomTasks table and insert the "Process Bucket Directories" item into it.
 
#Run the task, Instructions from the Instructions Category.This will essentially, create the IMAS.CustomTasks table and insert the "Process Bucket Directories" item into it.
 
#Drag the task "OnDemand MainPage" from the Create Panels category and then click on start.
 
#Drag the task "OnDemand MainPage" from the Create Panels category and then click on start.
 
 
http://www.international-presence.com/images/imas/ondemandtasks.png
 
http://www.international-presence.com/images/imas/ondemandtasks.png
 
 
#Drag the following tasks from the Runners category into the Queued tasks table and make sure they are running.
 
#Drag the following tasks from the Runners category into the Queued tasks table and make sure they are running.
 
 
http://www.international-presence.com/images/imas/queuedtasks.png
 
http://www.international-presence.com/images/imas/queuedtasks.png
 
 
#If you now go to http://localhost/imas, you should see the IMAS entry screen
 
#If you now go to http://localhost/imas, you should see the IMAS entry screen
 
 
If this does not work then you need to check that the HTTPServer is running along with the Presence Server.
 
If this does not work then you need to check that the HTTPServer is running along with the Presence Server.

Revision as of 09:32, 24 May 2012

Tutorial Pages > Internet Mail Archive System

Overview

This document describes the IMAS system and how it was created using Presence.

The IMAS System currently performs 3 functions:-

The Mail Service connects to an email account and downloads the attachments, based on conditions defined under the rules menu.
It will place each attachment in a separate folder in .\res\httpdocs\imas\scan\BasicDetach
The Rule Service will then process each of these directories by moving each attachment to a Network or AS400 location, 
or by running a task that will access the file through the variable ${localfile}.
The Task Service can also be used to run extra user defined tasks if necessary.

Presence Tasks

Queued Processes

Queue Scan POP
Queue Rule Action
Queue HouseKeeper - This is used to clean up the logs
Queue Run Custom Tasks - This is used to run any tasks defined in imas.customtasks
note We will also need to queue any tasks run by the rules

On-Demand Processes

OnDemand MainPage (alias imas.mainPage)

General IMAS Solution

Here we will take a look at the general IMAS solution and show how it can Scan attachments from emails and upload them to either an AS400 Folder a Network Folder or run a task.

The first screen here is the IMAS config screen.

IMAS is located in the Presence .\res\httpdocs\imas directory.

So with the both the Presence HTTP Server and the IMAS tasks running, we can go to:-

http://localhost/imas


The first screen we will see is the IMAS config screen.

http://www.international-presence.com/images/imas/editconfig.png

Here we can define an administrator email, who will get notified of any of exceptions that occur.

As each attachment is processed, the name of the folder that gets created is incremented.

We can also see the next folder number as well as define the name of the folder that failed attachments end up in.


Under the AS400 screen we can add and edit AS400 connections.

http://www.international-presence.com/images/imas/editas400.png


Similarly the Email screen allows us to configure which POP accounts we need to be read.

http://www.international-presence.com/images/imas/editemail.png


The rule screen lets us define what happens when we query a pop account.

http://www.international-presence.com/images/imas/editrule.png

Each rule is associated with one email account as defined in the previous tabs.

The action to perform can be one of:-

  • Upload To Network Folder
  • Upload To An AS400 (as defined in the AS400 menu)
  • Run A Task

Below the action type, we specify the Task or Network/AS400 folder.

The From Address, allows us to only process emails from a specific address (leave brank to process all the emails in this account).

In Attachment we can specify to only process emails that have a certain type of attachment.

In Filename we can specify to only process emails that have a certain filename match.

Subject allows us to only process emails that match this subject.

Convert XLS to CSV will convert any XLS attachments to CSV attachments.


This Mail Scanner Task can be stopped and started using the Mail Service Screen.

http://www.international-presence.com/images/imas/mailservice.png

When the attachments are taken from each email, a folder is created for each in the BasicDetach folder:-

http://www.international-presence.com/images/imas/basicdetach.png

Note that there are 2 files in each folder as one is the attachment and the other is the trigger folder (imasready).


The Rule Service performs the required action on each of the folders created from the Mail Service.

http://www.international-presence.com/images/imas/ruleservice.png


We can find a list of any emails that failed to be processed in the Email Exceptions Page

http://www.international-presence.com/images/imas/mailexceptions.png

From here we can email ourselves a copy of the report.


We can find a list of Rules that failed to run in the Rule Exceptions Pages

http://www.international-presence.com/images/imas/ruleexceptions.png

From here we can email ourselves a copy of the report.

Adding A Custom Task To The Solution

Here, we will study the solution delivered to DAS to give us an example of IMAS in action using Custom Tasks.

The requirement for DAS was to scan an email account for emails containting attachments that can be one of four types:-

  • HomeFixed
  • HomeDelimitted
  • MotorFixed
  • MoterDelimitted

Attachments, can also come through as zipped, in which case we needed to get Presence to unzip them so that it can then treat them as standard attachments.


As an example we will send 1 email that contains a zip file containing one of each file (HomeFixed,HomeDelimitted MotorFixed,MoterDelimitted) as well as an individual file MotorDelimitted.txt and another file Wrong.txt which contains invalid data.

When the Mailservice runs each of these files will be detached and placed in:-

res\imas\scan\basicdetach folder, each in a separate folder.

In this case we have configured the rule to tun the task Move Folder Into Bucket.

This task actually opens each file and decides whether it is:-

  • HomeFixed
  • HomeDelimitted
  • MotorFixed
  • MoterDelimitted

And places it in the corresponding scan directory. If it can't decide it places it in the Unknown directory:-

http://www.international-presence.com/images/imas/buckets.png

We then need to process each of the 4 directories, to do this we added a Custom Task, Process Bucket Directories

The Custom Tasks need to be added to the table imas.customtasks, SQL helper nodes for this can be found in the Instructions task.

The SQL statement Select * from imas.customtasks should produce the following result:-

http://www.international-presence.com/images/imas/customtasksselect.png

For each type of directory, it will populate a table on the as400 (from the file), and then run a program call on the as400, after which the file will be deleted.


To run this Custom Task, we need to go to the task service, select it from the combo, then hit start.

http://www.international-presence.com/images/imas/taskservice.png


Once Again the exceptions for this task can be found in the Task Exceptions screen.

This shows the as400 table (which should be empty) and any unprocessed folders.

http://www.international-presence.com/images/imas/taskexceptions.png

Installing/Moving IMAS to a new location

  1. Import the IMAS.xml into a new category on your Presence System.
  2. Copy the imas folder from the Presence \res\httpdocs\imas directory to \res\httpdocs\imas on the new system
  3. Run the task, Instructions from the Instructions Category.This will essentially, create the IMAS.CustomTasks table and insert the "Process Bucket Directories" item into it.
  4. Drag the task "OnDemand MainPage" from the Create Panels category and then click on start.

http://www.international-presence.com/images/imas/ondemandtasks.png

  1. Drag the following tasks from the Runners category into the Queued tasks table and make sure they are running.

http://www.international-presence.com/images/imas/queuedtasks.png

  1. If you now go to http://localhost/imas, you should see the IMAS entry screen

If this does not work then you need to check that the HTTPServer is running along with the Presence Server.