Bring Compliance-Related Capabilities into the Email Authoring Cycle
Site Map
Today,  financial services companies and others typically are  seeling to attain email compliance through "back door"approaches.  However,  within today's "state of the art," there are few, if any protections against a "smoking gun" email getting out the door.

There are three ingredients to the typically approach to email compliance.  First, to capture emails,  the bank or other institution implements comprehensive email retention. Second, to make rentention useful, they implement  indexing  to bring some order the  mountains of retained mail. Third, they implement filtering to detect potential regulatory violations and to report these possible problems to the sender's management or to other reviewers.  However, some companies have given up on the practicality of using filtering to block outgoing email, because filtering generate too many false positives, while false negatives (email that present undetected regulatory risks) go out in any case.

Fourth, as a last, perhaps forlorn  hope, companies automatically apply disclaimers to every outgoing message - often saying something to the effect that "the bank does not know what this person is saying or offering in this email and has no intention of honoring his or her commitments ..."

All of these steps are necessary, but not sufficient.

The better way is to address regulatory issues is to also work preventatively from the "front" - to embed regulatory-related features into the email creation process in a way that engages the user and that provides efficient guidance.  To accomplish this goal, "horizontal" services like email need to acquire "vertical" system features.  The term "vertical" is indicative of purpose-specific logic and data.

A company that has, for example, a "vertical" quote system can be assured that the quotes generated conform to whatever rules and data references. The user enters appropriate parameter values, and the logic and assciated data generates a compliant quote. In contrast,  an email can have any content the user chooses to generate. Therefore, the way forward is clear. For regulatory- sensitive email creation processes, incorporate some "vertical" application features so that essential rules are enforced at the point of creation.

On the other hand, what is also clear is that trying to "verticalize" email in its entirety is both expensive and unattractive, because vertical applications are slow to be delivered, hard to use and hard to update.  Indeed the present regulatory compliance problems with horizontal applications like email and spreadsheets are the product of defeatism and consequent neglect of vertical applications.

Successful, long haul compliance will require convergence of the horizontal and vertica application capabilities in easy to deploy offerings. SAP's Medicino offering might be illustrative although the vertical application underpinnings may still be less than agile. Given enough time, that convergence can be achieved by implementing "Business Process Management" (BPM) systems, but  to address compliance matters promptly there needs to be some early melding of email with vertical applications and rules-oriented workflow.


Many companies are struggling to achieve email "compliance"  - what cannot be said or must be said, and when it can be said. Although not the subject of this page, another manifestation of the same problem is the effort required to address the Sarbanes-Oxley compliance issues posed by heavy reliance on spreadsheets.

Although the PC revolution has ended or at least matured, the consequences are still very much with us.

Beginning in the late 1980's and early 1990's, the typical corporation developed two distinct Information Technology (IT) environments (more if it was involved in engineering or manufacturing). The business systems environment is termed "vertical," and it includes software that performs sales order processing, inventory management, payroll, general ledger and many other functional business processes. The term "vertical" was adopted because these systems were aligned with organizational "chimneys" - for example, payroll people could used payroll systems, accountants used the general ledger system.

The other software category - "horizontal" - was made up of "end user" personal productivity applications that could be used across the company - starting with word processing and spreadsheet packages later followed by email and presentation packages. The price per unit of processing power on a mainframe or minicomputer was much higher than on a PC, so most "horizontal" applications migrated to PCs - a trend accelerated by the user appeal of the graphical user interface (GUI) - economically supportable only on the PC.

Of course, those same PCs were too limited to support large, multi-user vertical applications, so like continents drifting apart, the horizontals went one way and the verticals went another. Also, horizontal applications as well as PCs became consumer products, so the horizontal world has enjoyed huge economies of scale and wide acceptance.

Conditions Today

Today, things are not all that different.

Although email began the decade of the 1990's as the least pervasive of the horizontals, one recent estimate is that 45% of the computerized knowledge base of a typical large company is in its email repositories. The fact that email is now a centerpiece of regulatory and litigation "data dragnets" merely reflects its position as being central to the communications of the corporation.

Vertical applications also have progressed, primarily in expanding their functional footprint.  Companies that in earlier days would have sales, purchasing, and invoicing systems used by clerks now have far wider functionality - customer relationship management systems, sales and distribution systems, production forecasting and planning systems, maintenance planning, etc.

Still, use of vertical systems has tended to remain narrow and transactional.  For example, a salesperson might use a vertical application to input or review a sales order or quote, but most of the substantive and relationship-building  process leading up to the order or contract - initial discussions, back and forth inquiries and negotiations, etc. would typically take place using "horizontals" - especially email - whether in isolation or as a means to transmit attachments.

The verticals remain more expensive and difficult to deploy and put into use. It takes a lot of training and coaching to prepare new users to use their vertical systems, and prospective users are unlikely to be trained in advance through "at home" or in school use. In contrast, there is wide familiarity with "horizontals" - notably Microsoft Office because Office is also a consumer and SOHO product..

Indeed, shortcomingsand rigidities  in vertical applications infrastructure sometimes creates aberrant user behavior - and perhaps regulatory violations as well. In a recent blog run by CIO Magazine, the writer described customer service situations in which a customer got the repairs needed and, in a separate incident, customers received the refund deserved, but the feature-level shortcomings in the suppliers' vertical systems required customer service management to authorize some field expedients. As a result, the transaction entries into the relevant vertical systems bore little relation to what really happened or why. The writer suggested that the "cure" is wider use of "end user" development tools so that, in my terms, the horizontal inmates take over the vertical world, but the odds of end user development being regulatory-conformant is pretty low.

Regulatory Impact

From a regulatory perspective, what happens when a "system of record" becomes a "sometimes" system of record? Also, supporting paper trails have eroded, not just for lack of printing, but because a dialog that in the era of typewriters and telegrams might have been three or four document exchanges now may occur as dozens or even hundreds of one or two line emails.

The SEC and others concerned with financial regulatory enforcement have concerns not only with a company's internal processes, but with its external communications, so the trends described above present some knotty enforcement problems.

One can paint scenarios - hopefully hypothetical - in which an email contains, for example, both an offer to sell securities and a commitment to buy them back - at a later date and at a guaranteed price. The sender's intent would be perhaps to park losing stocks, inflate cash, or otherwise violate corporate governance requirements and perhaps the law. Given that email is an end user and personal productivity tool, one would look in vain to the vertical systems to help prevent or track such departures.

Regulatory agencies (and, in the case of Sarbanes-Oxley, Congress) are not ones to suffer in silence or to bear the burden alone, so today companies are required to incorporate appropriate governance and auditability to their records-keeping and communications processes. Email compliance is therefore a very hot subject.

Although complying may be more or less difficult from company to company, the strategic problem facing everyone is that the horizontal and vertical "continents" drifted apart and, at a minimum, the gaps need to be bridged.

Closing the Back Door

Today, a lot of thought, effort and money is being spent to bring email-based communications into compliance. Interestly, some financial institutions are pushing some regulatory sensitive communications off the cell phone onto the email channel because as tenuous as email compliance capabilities may be, emails are easier to archive than cell phone traffic.

One necessary part of the email compliance solution is improved retention - to capture and store all relevant email traffic. To make retention useful, it is also necessary to index email so that compliance practitioners inside or outside the company have some hope of finding the violation needles in the huge haystack (without, by the way, violating other laws and regulations concerning privacy, pre-release of financial data, etc.).

A more proactive measure is to put servers to work scanning emails as they transit gateways. The scanning can detect and, at least in theory, shortstop messages that based on text searches or other factors appear to violate rules of governance. As many readers are aware from similar use of technology to detect and filter email spam, there are limits to what can be accomplished by machines. Therefore, the implementers need to provide for human intervention to handle the false positives - messages that contained some "magic words" suggesting a violation, but that in fact did not violate regulations and need to be sent on their way.

The filtering process also creates false negatives - messages that in fact violate rules of governance, but escaped automated detection and went out the door. In tightening the filters to constrain false negatives, often one impact is to create more false positives, so there are significant tradeoffs.

Although closing the back end with these and other measures is undoubtedly necessary, the better solution is to focus on the front end of the process.

Working from the Front

The proactive solution is to create and use vertical applications as the source of sensitive emails, so that the email will conform to required processes and regulations. Such an application needs to provide much the same convenience as normal email creation process, along with "calls" to essential data stores (e.g., lists of appropriate trading partners) andto  "rules" and workflow. However, all this is more easily said than done.

Success requires overcoming an enormous amount of doubt, because the vertical systems world tends to be slow and its services cumbersome.  Design and development is slow and risky, initial rollout is slow, expensive and risky, lifecycle maintenance is often far too slow to keep up with changing conditions, and at the end of their lifecycle vertical systems often do not die with dignity, but go through long periods in "walking dead" state.

The need is to create "vertical" (specialized) workbench environments that blend vertical application disciplines with horizontal application user friendliness and adaptability. Today, someone building such a workbench can invoke the same tools that end users would ordinarily use to create an email or other document. Such an application would need to be developed in a RAD (rapid application development) environment, because the context is much more about agility than complexity. It also would not be a monolith, because rules regarding consumer lending and rules regarding investment bankinhg are different and one size does not fit all. On the other hand, the data bases or databases would need to be coordinated because some regulatory matters cross "stovepipes."

The crucial point of this page is that there is a better way than trying to ferret out "bad" messages after they have already been created and released for transport.