The JLARC Review: Looking Beyond the Statistical Surface (Part 3)

Posted on April 24, 2009


The one common element, besides the irony that both the Government of Canada’s and Virginia’s programs were launched in 2001, is the paucity of either statistical or empirical proof to support the current mandates. For example Virginia’s Joint Legislative Audit and Review Committee’s (JLARC) reference to a survey, in which 95 out of a possible 46,000 stakeholders responded, as one of several reasons for their review is difficult to grasp.”

From the April 22, 2009 Procurement Insights Post “Synchronization Versus Compression Should Guide Government Procurement Policy (200th Post)

In the world of sports there is an old saying that “statistics are for losers.” It is based upon the fact that the team with the best numbers relative to overall season points and position in the standings, goals for or against, batting average, even the highest payroll etc. does not necessarily translate into a championship season.

We have certainly seen countless examples of this such as in hockey where the 8th place team upsets the 1st place team in the opening round of the playoffs. Or in football where the Buffalo Bills had what it took to get to the “big game” multiple times, only to come away empty handed. Even the classic baseball story of Casey At The Bat provides an example that past performance doesn’t ensure future success . . . especially at the point in time when it is needed most.

Because of these examples, as well as outcomes in other facets of everyday life, taking the time to look beyond the numbers can cast both an interesting and previously unseen light on a situation that can ultimately better serve the collective interests of all parties to a particular issue.

As previously indicated, I have and at this point continue to be a strong supporter of the eVA program. Certainly my extensive research and ongoing analysis has solidified the validity of that position over the past two years. However, this does not preclude stepping back to look at eVA through a different set of eyes so to speak. And it is for this reason that I am in full support – at least in principle – of the JLARC Review.

The problem I have is the overly narrow scope of the mandate’s focus, and the methodology that is being employed to assess eVA’s viability. One example is the survey results which reflect the numerical and subjective opinion of a disturbingly low number of respondents. Working with a fraction of the resources of that which is available to the Commonwealth’s legislative analysts combined with somewhat limited access to stakeholders, my survey has already garnered a better response rate in terms of percentage ratios.

That said and keeping in mind that we have a long way to go relative to building the critical mass that is necessary to make a truly informed assessment, the overall results to date are certainly interesting. Specifically, 63 percent of respondents believe that eVA is a benefit to Small, Minority and Women-owned businesses. While this provides a degree of clarity from the “yeah” or “nay” perspective, what does this early result really tell us?

The Hidden Points of Support and Contention

The following is an in depth analysis of the poll results, which focuses more on the respondent than the responses. This in and of itself should lead to some interesting questions.  (Note: Use the following LINK to vote.)

Age: 71 percent of all responses come from individuals within the 35 to 54 age range. Of this group only 40 percent felt that eVA was a benefit.

In the 55 plus range, 100 percent feel that eVA is a benefit.

Does this reflect a “wisdom comes with age” axiom, or does it indicate that eVA is gradually losing touch with the current day approaches and technologies that are necessary to ensure its ongoing relevancy?

Gender: 40 percent of all respondents are female, of which an equal percentage of women were both for and against the program in terms of its benefits.

The male population, which represents 60 percent of respondents, shows that 67 percent believe that eVA is a benefit to suppliers.

What is telling here is that one of the key tenets of the program is the effective engagement and utilization of women-owned businesses. With only 50 percent of the women who have responded to date believing that the program is a benefit, is perhaps an indication that more work needs to be done within this important sector.

Job Function: 67 percent of all respondents hold a sales position with their respective organization. Only 50 percent of these individuals believe that eVA is a benefit to Small, Minority and Women-owned suppliers.

100 percent of the remaining responses, which are from individuals who are more involved in a “creative” function within their organization such as marketing, believe that eVA is a benefit.

What this tends to demonstrate is that eVA is conceptually sound, but lags in terms of acceptance from a practical, everyday standpoint.

Company Size: A combined 28 percent of responses have come from individuals within the Enterprise and Large business sector. Perhaps surprisingly, especially given the fact that the program is focused on stimulating SME development and engagement, eVA garnered 100 percent support from this segment.

Medium business interests account for 29 percent of all respondents, with support regarding the question of the program’s benefit to the SME community being equally divided (50/50).

As expected, the majority of respondents, 43 percent, come from the SME sector with 67 percent expressing their belief that eVA has in fact been a benefit to their businesses.

The above results create more questions than they answer. For example, what are the reasons for the broad acknowledgment of the program’s benefits to SMEs by the enterprise/large business sector? Does this reflect an opinion that based on a decline in their own (Commonwealth) revenue numbers, the assumption is that business must be going to small business suppliers, and are therefore perceived as being a benefit by that group? What impact does this then have in terms of the Commonwealth being able to leverage volume discounts and sustainable savings?

What about the medium-sized business community? Does the equal division of opinion highlight the fact that this classification of supplier is the “forgotten” middle child of the Virginian supply base.

Finally, while the majority (67 percent) of SME responses tend to support eVA’s effectiveness, is this enough of an endorsement for the program? After all, the 33 percent of SME suppliers who do not believe that eVA is a benefit doesn’t reflect an isolated “pocket” of resistance. It is still a sizable enough number to prompt further investigation.

Job Title: C-Level Executives and Vice Presidents represent 11 percent of respondents to date. Of these, 100 percent do not believe that eVA is a benefit to Small, Minority and Women-owned businesses.

44 percent of those who have responded to the survey come from the management ranks. Of these, only 25 percent believe that eVA is a benefit, while 75 percent share the same sentiments with their more senior counterparts relative to the program’s effectiveness.

The remaining 44 percent represents the front line work force, from which eVA receives unanimous (100 percent) support.

These results reflect the universal disconnect that plagues business in general, and for the most part has contributed to the high rate of initiative failures – for the uninitiated, 85 percent of all eProcurement initiatives in both the public and private sectors fail to achieve the expected results. (Note: for those who might be interested in a more detailed understanding of initiative challenges, including actual case references, check out my white paper “SAP Procurement for Public Sector.” It is an “eye-opening” study to say the least.)

The net result relative to Job Title response is that from a practical utilization perspective, eVA works and works well. In short, it overwhelmingly meets the real-world functional requirements of the user community in the key areas of cycle-time reduction and ease of use. This of course removes the usually insurmountable barriers associated with user adoption and program compliance.

A Long Road Ahead

As indicated, the results to date are not yet significant enough to form a reliable and lasting consensus regarding eVA’s on-going viability.

What they do provide is the basis for viewing the legislative exercise through a broader and more diverse lens of understanding. In essence, stimulate the questions that might not have been asked in an effort to shape the investigative process to be more in line with an agent-based versus equation-based methodology.

Based on the above, I cannot say with any certainty that the artificially imposed report release date of July 13th will serve the collective interests of the Commonwealth regardless of which side of the issue one happens to be on. What I am saying is that there is still a long ways to go before a sound and reliable decision in terms of the re-working of Virginia’s procurement policies can be done with any level of confidence.


eVA Poll and Corresponding Radio Broadcast

We have just posted a Procurement Insights/PI Window on Business Poll that asks the question “Is Virginia’s eVA program a benefit to SME/minority & women-owned suppliers?

Click on the above question and take a few moments to cast your vote. Please note that the poll has been created through the LinkedIn Business Network. If you are not a network member but want to vote, simply follow the on-line prompts to quickly set-up a profile. There is no cost to join, and you will also gain access to a number of tremendous resources including the newly established “Public Sector Supplier Forum.”


In mid-June, the PI Window on Business Blog Talk Radio Show will do a special broadcast on the JLARC legislative review, where we will accept calls from all sides of this important legislative process.

Visit the PI Window on Business Show’s main page for broadcast details.


Listen to Jon Hansen (PI Window on Business) on BlogTalkRadio talk radio
PI Window on Business is a subsidiary of Procurement Insights Publishing.


The Procurement Insights Blog reaches more than 300,000 syndicated subscribers each month worldwide.

Use the following links to learn more about becoming a sponsor for the Procurement Insights Blog and the PI Window on Business Blog Talk Radio Show, or contact Jennifer Cameron at, (819) 986-8953.