The mission at the Minnesota Department of Human Services (DHS) is to help Minnesotans meet their basic needs so they can live in dignity and achieve their highest potential. At DHS, they are also committed to accountability and continually improving performance.
In an effort to be innovative and accountable, DHS began implementing tools for auditing and monitoring many of their systems and data sources. Traditional tools such as Oracle Audit Vault™ and HP ArcSight™ were among the many put into production use.
Over time, data sources increased in number and in size, standards grew from many different directions and things just became more complicated and expensive.
The CIO decided that based on the level of engagement Collier IT had with Oracle Audit Vault™ and Oracle Databases within DHS, he would work with Collier IT to investigate the potential of a “compliance as a service” model that could be affordable, sustainable and flexible enough to be adopted by many.
Collier IT engineers and IT architects utilized a mind-map approach with our suppliers and subject matter experts for best practices, methodologies and disciplines around putting together a flexible data environment that would provide a high level offering around audit, compliance, process improvement, detection, analytics and alerting.
Because of the diversity in governance sources in play, Collier IT presented several different viewpoints for gathering, cleansing, managing and utilizing data. The DHS staff added their insights each time and after 18 months of vetting, a decision was made to build out a solution that would be distributable and supportable without the costs associated with licensing a pre-defined product.
The tools selected were an Oracle pre-engineered Hadoop cluster utilizing the Cloudera Hadoop Distribution paired with programmers experienced with the Cloudera toolsets along with knowledge sharing and training from Collier IT’s certified Oracle and Cloudera instructors. A statement of work utilizing an Agile iterative approach was followed to ultimately deliver actionable alerts from an initial ingest from five diverse sources of streaming data.
- Hadoop environment set up
- Data Ingest and cleanse
- Data organization
- Secure to governance standards
- Produce actionable alerts
The solution is implemented, certified and ready for production use-case deployment and is currently free from fees for anything beyond maintenance for linux, Cloudera and hardware.