The Difference Between Metrics and KPIs

Thursday, March 16, 2017

By Dr. Michael Tal, Managing Director, BusyLamp

The biggest challenges faced by legal departments according to corporate counsel are improving operational efficiency, ensuring regulatory compliance, alleviating budget pressure, delivering despite a staff shortage and making better use of technology. So, in a nutshell, doing ‘more with less’ is an overriding objective. Often, corporate counsel often has a gut feeling that everyone in the department is “doing more with less”, but struggle to quantify the “more” and the “less”.

All the above-mentioned challenges could to a large extent be reduced by using legal analytics. Existing data can help to measure these items and drive quantifiable improvements. In addition, it allows the legal department to speak the language of company executives and senior management – i.e. the language of metrics and Key Performance Indicators (KPIs).

To obtain high-quality, trustworthy analytics, one must feed it with good data. There are several factors that determine quality data. With regard to legal spend management, these are for example: categorised matters, Uniform Task Based Management Systems (UTBMS) task-coded line items, robust line item descriptions and line items that are not block-billed.

Law departments have an abundance of available data, like historic bills from their outside counsel, which contain a wealth of valuable information, if they could mine it. This requires a structure and using legal analytics platforms is a good idea. These platforms can structure, aggregate, organise and compare data in meaningful ways to make it truly actionable in the form of metrics and KPIs.

To speak bluntly, metrics in organisations tend to be what can be easily measured. So often decision makers discount the metrics that don’t appear to achieve the outcome they desire at the least possible cost. Nevertheless, in order to convey the resounding impact of process innovation and improvement through the use of applied metrics on the legal department, it’s important to define success factors, limitations and choose KPIs that align with the established strategy. This makes the relationship between a KPI and a metric tangible – a KPI is simply a metric that is tied to a target.

A metric is generally understood to be a standard of measurement and that can be used as a way of quantitatively assessing the efficiency of performance with respect to a particular process. In order for a metric to be useful, however, there must be full agreement in advance as to how the relevant data will be collected, organised and displayed. Metrics cannot be used in isolation and their results must be evaluated against pre-agreed standards. This exercise is referred to as “benchmarking”, post which a large number of “performance indicators” can be identified. The focus must be on those that are truly “key” – meaning that KPIs should have a sensible and sustainable architecture, which aligns with the company’s overall strategy and are easily understood by stakeholders while being limited to those areas that can create the most value for the organisation.

There is no “right answer” for what metrics a legal department should use and the KPI programme must be implemented, but there are some questions that can help drive the decision-making process:

  1. Why are the KPIs being developed? For example, is it to demonstrate the value of the legal department; or just track trends?
  2. What are the department’s goals? What are the company’s overarching goals? Is there a strategy you could review?
  3. What aspects of performance are of value to leadership (company, law department, business units)?
  4. Does anyone have extensive experience of developing and implementing KPIs within the organisation?
  5. Are metrics being tracked in other areas? What do those look like and what is the process for evaluating and measuring them? What were the lessons learned?
  6. What processes and tools are in place already that could help with measurement? What processes and tools would need to be implemented and what does that effort look like?
  7. Where does the KPI programme fit with the priorities of the department?

When determined following a diligent process, metrics can help incentivise the legal department and they can be embedded within the workflow. KPIs, on the other hand, should be used to establish goals and objectives for further improvement. Note however, that the goals should not simply be to achieve a “better score”, but rather be supported by specific “next steps” that can be understood, implemented and measured. This approach will enable the department to truly strive for and achieve definite improvements.

Wednesday, March 8, 2017

Information Security – The Past, Present and Future

Guest Blog by Niamh Muldoon, EU Information Security Director, DocuSign

“It will always be about protecting data”

The Past:

In the past, Information Security frameworks were inwardly focused organisationally, predominantly consisting of policy, standards and procedures that the organisation adhered to. The majority of data and information was stored in-house within the business technology perimeter. Data and information assets existed in both electronic and physical formats. The ability to technically share data and information was very limited and when permitted, subject to stringent Information Security Framework controls. Technology departments’ management tooling supported security teams with the administration of tooling, protecting the perimeter and businesses’ data.

Businesses that understood data and classified their electronic data were considered to be mature; they had appropriate controls in place, and their data was accessible internally and externally. The perimeter-to-perimeter connectivity was mainly focused on a one-to-one relationship, including underpinning network topology using leased private telecom lines progressing to the virtual private network concept.

The cyber-security threat landscape of the past was very different to the present day. In 1988, we saw the first piece of malware being introduced. Robert Morris created the ‘worm’ type of malware. The ‘Morris Worm’ impacted over 6000 computers and it’s said that it would have cost $96 million if it hadn’t been stopped. Not too many years later, we had the first well-publicised Distributed Denial of Service attack (DDOS).

The Present:

Fast forward to 15 years later to where we are today, and really the only way to say this is – the Internet has exploded. We all, in our personal and business lives, have embraced digitalisation and digitisation using Internet-based technologies.

To support this digital transformation, businesses have transformed to a ‘perimeter less’ ecosystem and Information Security Framework. With the starting point of the Framework commencing at the law level, the required laws that businesses need to adhere to for the data they manage and use, including global and country-specific laws, need to be considered. The next level up is the data protection regulations for the jurisdictions of operation – adding industry compliance standards that have a data component and industry best practice standards. These four levels all support businesses to digitally operate globally. Businesses need to internalise these requirements into policies, standards and procedures for data management that the internal organisation must then adhere to.

Operating at a global level using Internet/cloud based technologies means that businesses need an Information Security Framework that not only protects the Information Asset categories from cyber security attacks, but also supports safeguarding the authorised use of those assets including their transfer across geographical borders.

The requirement for understanding data, classifying it into Information Asset types and protecting them has become even more granular. Today’s cyber security threat landscape is very different to the past. The means, opportunity and motive to attack have not only grown, but have advanced.

There were a total of 2,260 reported data breaches in 2016. Motives for attacks vary, but because hackers have the means and opportunity, it’s become far too easy to be a target for attack with the highly likely result of a data breach ensuing.

The Future:

We won’t know what the future will look from a technology advancement perspective, however we do know that data transformation to digitalisation will continue or be complete. Pro-actively managing and protecting Information Assets types is proven to be the foundation of success, enabling businesses to reduce the risk of security breaches and associated cyber security attacks – this will remain fundamental in the future.

At DocuSign, we have on average 130,000 new users joining the DocuSign Global Trust Network every day and our user-group is spread across 188 countries including regulated industries.

DocuSign has chosen ISO 27001 to be the core for its Security Standard model and it allows customers to take ISO 27001 results and map them into their vendor management programmes. DocuSign has also invested heavily in a cyber-security programme, investing 3 million into the Centre of Excellency. The Binding Corporate Rules programme is underway and the company continues to provide assurance with Compliance certifications.

About Niamh Vianney Muldoon

Niamh has over 15 years’ experience as an information security professional. Prior to DocuSign, she was the EMEA Technical Compliance programme manager at Workday where she established and grew the organisation’s EMEA Privacy and Compliance team. Through her Information Security programme manager role at the Irish online eGaming company, Paddy Power Betfair, she transformed the team from being an IT Security function to a mature Information Security function. Niamh spent over 12 years at the Irish Financial Institution, AIB; where she commenced her Information Security career and had many successes in relation to technology risk assessments frameworks and establishing Enterprise Information Security & eDiscovery functions. In her spare time, Niamh likes to give back to the Information Security Industry working with (ISC)2 sitting on the global scholarship review panels. She I is also a part-time evening lecturer in cybersecurity at the National College of Ireland.

Wednesday, March 22, 2017

Analytics and Machine Learning Can Help Detect Cyber Threats

The challenge of cyber security is laid bare when RSA, one of the world’s preeminent security firms providing solutions to address cyber threats, is unable to prevent a social engineering attack on itself. This of course happened in 2011, but given the frequency with which companies are breached today, no enterprise can be a 100% sure of preventing an attack. Why? The real threat lies ‘within’ the organisation. Employees are the weak link and hence the easy target for cyber criminals to exploit in order to get hold of sensitive data.

So, how must a law firm defend against its own employees – the very people who it must provide data access to – without ‘getting in the way’ of their work? Furthermore, it’s not always apparent that the organisation is hacked. The average time that an attack went undetected in a network in 2016 was approximately 150 days.

Best practice suggests that in the immediate aftermath of an attack, firms must:

  • Understand how the breach occurred in order to immediately remediate any deficiency that was exploited

  • Quickly identify where the firm’s exposure is, especially which clients are impacted

  • Recover lost data

  • Notify the relevant regulators and the ICO of the breach. This will become even more pertinent with the new EU General Data Protection Regulation that comes into force next year

A document management system (DMS) plays a key role here – after all, this is where the firm’s ‘crown jewels’ reside and so protecting the sensitive information within it is essential. Today, technology has advanced and security is no longer simply about complex passwords. Adoption of analytics and machine learning is essential to proactive data security and loss mitigation.

Firms that deploy iManage Work, a next generation DMS, are now able to automatically profile the typical behaviour of users and use this information to identify potential social engineering attacks and take pre-emptive action. This is provided by the recently announced Threat Manager module, which uses machine learning to automatically generate a digital ‘finger print’ of all employees based on their individual history of interaction with data in the DMS. Any deviation from this baseline is a good indication of anomalous activity and warrants investigation. The behavioural analytics used by Threat Manager is significantly more advanced than simple threshold reporting; and by understanding the matters, clients and practice areas relevant to the firm’s personnel, it can identify a hacker using stolen credentials to view even a small number of files as being an anomaly, and flag this for immediate action. In fact, an interesting finding during the testing phase of Threat Manager was that it is also able to predict with high accuracy, personnel who are likely to be leaving the firm. This is often a concern for firms due to potential loss of organisational and client IP.

In addition to threat detection, Threat Manager also provides capabilities to assist with investigating the timeline of the attack, leveraging the comprehensive audit trails within the DMS to identify precisely which matters and documents have been accessed. The tool is simple to use and means this crucial capability can be removed from the IT department and given to a more appropriate function such as the Risk and Compliance team. Those responsible regularly receive reports on alerts detected with corresponding scores based on the threat level, enabling them to take immediate action to prevent any further wrongdoing.

Given the nature of the threats faced, adoption of behavioural analytics and machine learning is going to be a key tool for firms wanting to take a proactive stance on data security and providing the assurance their clients are increasingly looking for.

Note: As an iManage partner, the new Threat Manager module for iManage Work is available from Ascertus. Please get in touch via for more information on the module and/or a demonstration of the product.

About Frank White

Frank has worked in legal IT for over 25 years, having occupied a number of technical roles through to IT Director of Ince & Co, responsible for delivering a professional yet personalised IT service and cost-appropriate IT solutions to a diverse global practice. Frank joined iManage in February 2015 as a Subject Matter Expert to help iManage customers realise the best return on their investment in their iManage products, and to enable the company to deliver better products and create more value.