MarketWorks working for you in your market

Overview of Transaction Assurance

Our transaction assurance offering provides the guidance and direction that assures that services contracted by clients are delivered according to expectations and contract. We ensure good governance is and that the client achieves the benefits envisaged by the transaction.

Benefits of MarketWorks Advisory Transaction Assurance offering

• Ensuring compliance with the agreement - MarketWorks Advisory develop a checklist from the agreement and assist the vendor to set up the vehicles and processes to ensure compliance.
• Risk analysis - MarketWorks Advisory facilitate client and vendor workshops to identify and prioritise risks and develop mitigation strategies. We will ensure that the vehicles and processes to monitor progress on mitigation action are implemented.
• Deliverables definition - MarketWorks Advisory ensure that Tender or solution deliverables are defined properly and that delivery dates, acceptance procedure and arbitration processes are clear and realistic.
• Governance structures and processes - MarketWorks Advisory review the governance structures and processes and advise where weaknesses are evident and improvements can be implemented.
• Assistance to programme managers / executives / sponsors / owners - MarketWorks Advisory will assist the client programme manager with any governance issues during the initial phases, whether that relates to resources, plans, structures, etc. Should we be required to, we will serve on steering committees and other management bodies.
   
  Who should use this service
  This service will improve the chances of success of any large or complex implementation. Project sponsors or programme managers who wish to set up a project for success right from the start will benefit from this service.
   
  How Transaction Assurance works
  After agreement is reached and before the planning phase gets underway, the Marketworks Advisory team will analyse the critical governance issues for the project. Thereafter they will engage with both client and vendor to set up the required controls, structures and processes on a project deliverables basis.
  MarketWorks RFP development and Tender evaluation Process

 FIND OUT MORE...

Tender Background and Context

The role MarketWorks Advisory play in the oversight of the tender process usually starts at the requirements definition phase, prior to the tender request development and publication. The reason for this is that the success of the evaluation process is closely linked to the format of the Request for Tender and the nature and content of the questions themselves.

The MarketWorks methodology of tender evaluation strives to provide the highest degree of confidence in the final choice. This includes the level of defensibility of the process and the outcome. It stands to reason that the format and content of the RFT could adversely affect the outcome, regardless of how robust the evaluation process itself is. If the right question were not asked, the information on which the decision is based may be incomplete.

The following paragraphs describe our approach and the rationale behind it.

Objectives of the MarketWorks Advisory approach to Tender development

The evaluation methodology is driven by an overarching principle of good governance and the objectives can be described as:

•    Speed of process

•    Quality of evaluation

•    Fairness and equitability

•    Consistency

•    Transparency

•    Control, in terms of both manageability and, more importantly, defensibility Impartiality

•    Clarity of outcome, informing a clear decision

Objectivity

MarketWorks is the sole representative of Datamonitor in Southern Africa. Datamonitor and its subsidiaries, Butler Group and Computerwire provide independent research, opinions and comment on technology and the IT industry and can consequently not have any interest in or affiliation with any vendor. MarketWorks is equally serious about maintaining complete independence from any vendor product or group.

Our opinions are completely objective and based on experience and independent research only.

Tender Planning

During this phase the approach and the process will be clarified to the Project Team, roles will be clarified, responsibilities will be allocated, time-lines and goals will be established and critical decisions will be identified. An important part of the Planning phase is to identify risks and critical success factors and, as far as is practical, address them.

One of the most common reasons why large projects are stopped before they get properly underway is the lack of a clear and strong business case document. Every project reaches a stage when the enormity of the task and the expense becomes apparent and questions are raised. If the business case is not documented properly these questions can often not be answered convincingly and the project gets “shelved until further notice”. This is a risk worth considering in this case. However, we are not aware whether a business case document does exist or whether the Department considers it necessary. The drafting of such a document is not usually within the scope of this assignment. It is however highly advisable and preferable to have an approved business case.

Requirements Analysis

The two most important features of a good RFP is that

•  It must prompt responses that can be compared and evaluated; and
• It must be precise enough to ensure that the winning solution will be at least adequate.

 At the same time, however, being prescriptive about the solution creates the risk that new, innovative and cost-effective ideas are not considered.

While it is easy to compare the quality and value of tangible objects such as, for example chairs, when one sets out to buy services, software, systems and such, there is always the risk of a mismatch between expectations and actual delivery. The requirements analysis phase aims to establish criteria that will be clear, comprehensive and precise, without being prescriptive, in order to meet these basic requirements and reduce the risk of receiving less than was expected.

In practice, the logical sequence of events is to start by running workshops to define the scope and objectives of the RFP. A simple example is to answer the question, “what is the ideal contractual situation for the organisation?” If the organisation contracts with one vendor to deliver everything, the burden of managing many suppliers is reduced. However, how do you ensure that everything is provided at the best price? These kinds of questions are debated by the project team and every time a decision is made, the scope and objectives become more clear.

The first complete draft of the scope and objectives document form the framework to begin to define criteria and the questions that will show to what extent individual vendors can meet those criteria. The first draft of the Criteria Questions document is scrutinized for implications and gaps and can lead to refinement of the scope and objectives. This process may go through a few iterations before the requirements are clear and comprehensive.

Producing a Draft RFP/RFT

Once the Scope and Objectives are clear, the Project Team must produce the Background and Context documents that will inform the vendors on how to approach the proposals. This task is logically allocated to staff within the Department and is checked for quality and completeness by the MarketWorks team. MarketWorks will provide guidance with regard to the content and structure of these documents.

MarketWorks will draft the standard documents, such as instructions to vendors on the response format, delivery of documents, and so forth. MarketWorks will also draft the commercial requirements documents under the guidance of the Department and within the specific procurement rules and requirements. Many of our standard commercial requirements serve to ease contract negotiations and protect the Department against common risks.

There are several chapters in the document for questions to evaluate the vendors’ solutions against the criteria. The exact structure and approach to each of these chapters depends on the outcome of the requirements analysis and should serve to get the best result. Typically these would include a chapter on the quality, experience and substance of the vender consortium, a chapter on the design and architecture of the solution, a chapter on the implementation plan and process, a chapter on the support and maintenance requirements for the solution, etc. In addition, there could be specific chapters on such issues as sustainability, socio-economic implications and so forth.

The Pricing section of the document should be as simple and logical as possible. This is often an area that is difficult to compare as vendors tend to include many variables, conditions and options into their pricing proposals. A standard approach, such as total cost of ownership should be dictated and the conditions and parameters spelt out, leaving as little room for interpretation and optional extras as possible.

The Participation Goals or BBBEE section of the document can follow standard procurement requirements, but must be clear and logical.

The final document will be structured logically, with clearly marked sections and no duplications or contradictions. Each question will have a unique number.

Quality Assurance

Despite the need to speed up the process as much as possible, a quality check on the document before finalisation is essential. The main goal is to ensure that responses will be complete and easy to evaluate. Simple things like spelling mistakes can cause misunderstandings and lead to objections by vendors.

The final RFP document can be considered as a milestone should the project be divided into phases.

RFP/Tender Publication

The advertising and publication of the RFP is the responsibility of the organisation and should happen on the scheduled date. It is not essential to hold vendor briefing sessions, but should it be felt that there is a compelling reason to have such sessions, MarketWorks will play the role required of it by the Project Sponsor during these sessions.

The submission deadline should be realistic and driven by the size and complexity of the RFP more than by the urgency of the project. As far as possible, there should be no reasonable requests for extensions.

Vendor Questions Management

After publication, and up to a week before the response deadline, vendors should be afforded the opportunity to pose any questions about the content or format of the RFP. Answers to such questions should be carefully constructed and published together with the questions in such a way that they are transparent to all the participating vendors. Ideally the Department website should be the channel for this communication.

The answers to the questions should be agreed by the Department and checked by MarketWorks in order to eliminate the risk of scope or interpretation changes creeping in. The closing date for questions should be timed in such a way that answers can be published at least one day before the final deadline. It is important to answer all questions, even the silly and illogical ones, in order to reduce the possibility of vendor objections.

Evaluation Preparation

Between the date of publication and the response deadline there are a number of preparatory tasks to undertake. The selection of the evaluation team and confirmation of the sections of the document that each will evaluate can take place during this period. Evaluators are chosen based on their individual expertise and will only be required to evaluate the sections within their expertise. It is quite possible that there will be a need for general evaluators for sections where there are no specific expertise required. Evaluators are appointed and their availability confirmed for the evaluation period. MarketWorks will provide facilitation and guidance during this process.

Infrastructure and logistics preparations can be made during this period to ensure that the evaluation process can run smoothly and quickly. MarketWorks will assist with advice and guidance during this process.

MarketWorks will facilitate the establishment of evaluator guidelines for evaluation purposes. These guidelines are simply notes to the evaluators on the questions to provide some guidance regarding the important issues and the preferred options when assessing responses. This ensures focus and consistency in the evaluation process. The guidelines are provided by the project team and some of the evaluators or other experts that cannot be on the evaluation team.

An important task to undertake during this phase is to establish weightings for all the criteria. MarketWorks will facilitate weighting workshops with a panel of senior people within the Department. The members of this weighting committee may not evaluate any responses or influence the evaluators in their evaluation tasks. Internal Audit should be represented in the weighting workshops and the final weights should be lodged with them for safekeeping until the after the completion of the evaluation process.

The scoring metrics and definitions of metrics will be agreed and prepared for evaluators and the value model will be populated and prepared. Towards the end of this period, MarketWorks will conduct evaluator briefing sessions and prepare the documents and forms for the evaluation process.

Response Management

It is envisaged that Procurement and Internal Audit Departments will take custody of responses as they arrive. The Evaluation Management team will take ownership of this responsibility when the evaluation process is ready to begin. MarketWorks can assist with advice on procedure and security.

Evaluation Templates

Before the evaluation, the Project Manager will prepare all evaluator instructions and evaluation templates. The Project Manager will decide on the metrics scoring system to be used in the evaluation and will clearly define the meaning of each metric within the system. A typical metric scoring scale will be between 0 and 3, where 0 is a complete fail and 3 is a complete pass. Each number must, however be defined in as much detail as possible in order to facilitate consistency.

For example:

 “0/3 – Does not meet requirement:

• Vendor gave no response; or,
• The response was unrelated to the requirement; or,
• Vendor responded incorrectly; or,
• The response demonstrates a lack of understanding of the requirement; or
• There is no substance to the response; or,
• The substance of the response is without any merit.”

Where questions require a binary (yes / no) response, the definitions obviously will be less elaborate.

The Project Manager will then draft evaluator instructions and guidelines relevant to each question in the FRT. The Project Manager will prepare a timetable to monitor and check progress against as the evaluation progresses. Finally the Project Manager will arrange all logistical necessities beforehand to facilitate speedy evaluation.

Weighting of Criteria

The weighting of criteria represents the priority or relative importance that the strategic management team within the DoJ & CD gives to the criteria. For example, “Stability” may be more important than “Scalability” and will carry a higher weighting. The weighting exercise should be carried out before any responses are viewed and should not be disclosed to evaluators, in order to avoid any undue influence. The weighting must be done by a team of business and strategic client representatives. Weighting should be done down to the lowest possible level and rolled up to the main criteria. In this way the scores will be rationalized to align to the strategic priorities of the organization.

The exercise can be completed without carrying out any weighting exercise (all criteria are regarded as equally important), but this is likely to adversely affect the accuracy of the outcome.

Evaluation Process

The evaluation process uses subject experts to evaluate the different sections of the RFT responses, independently. Evaluators are not allowed to discuss the questions or influence one another in any way. In order to facilitate questions and clear up any confusion that may arise, Consultants serves as mediators and ensure that consistent explanations and instructions are given to all evaluators on all queries that arise during the process.

The responses are split into the categories: Solution (Technical); Price; and BEE and each of these sections are evaluated separately by different individuals. Each section is evaluated by a minimum of 2 individuals.

Security

Evaluation takes place on secure premises where passing traffic is kept to a minimum and the environment can be managed comfortably. Evaluators are not allowed cell phones, laptops or any other means of communicating with the outside world into the evaluation room, and no evaluation material or notes are allowed to leave the evaluation room.

Strict security is exercised and the process is made as defensible as far as is humanly possible. Minutes are kept every day of all incidents and each evaluator gets the opportunity to comment on the fairness of the process at the end of each evaluation day.

Evaluators are asked to sign non-disclosure and secrecy undertakings and to declare all interest in any of the vendors represented in the responses. Rooms and all evaluation material are kept locked when not supervised by the Project Manager.

Arbitration

The Project Manager will chair arbitration meetings to make decisions on any incidents during the process. An arbitration committee will also deal with all anomalies in the scoring. The evaluators involved in anomalies will get the opportunity to confirm or change scores resulting in anomalies. All proceedings are recorded and signed by all parties present.

Consolidation

When evaluators finish evaluating a section, their score sheets are handed in to the Project Manager who then enters all the scores into a spreadsheet for consolidation. Alternatively, if electronic scores sheets are used, these are transferred to a consolidation sheet. Evaluators check their score sheets against the spreadsheet to confirm that the information was transferred accurately.

The sections, Solution (Technical); Price; and BEE are still kept separate and a final score per vendor, per section is calculated. All calculations are checked by an independent person.

In our experience, simply applying the weightings to the sections and consolidating the scores per vendor gives a slanted outcome. To calculate the value for money factor for each vendor response gives a more meaningful result. The MarketWorks value model achieves this. The details of the mathematical method applied by this model can be provided in order to verify results. The use of this model is optional, however and not part of the assignment unless specifically agreed.

Outcome Assessment and Report

The outcome of the consolidation exercise provides a final ranking of vendor solutions. Evaluators will be given the opportunity to comment in writing on their opinion of the final outcome. Should the ranking not conform to the expectations of the evaluators, the scores will be checked and if necessary all information is rechecked and confirmed. Any errors or changes are recorded in the minutes that are kept during all evaluation proceedings.

Once a final outcome is produced, the report will simply confirm the outcome and the validity of the process and supply all supporting documentation. The report will not attempt to summarise or justify the ranking of the vendors in the final outcome. Should there, however be any relevant comments from evaluators, these will be highlighted. All relevant supporting information such as score sheets, calculation spreadsheets, daily minutes and evaluator declarations will be included for audit purposes.

 FIND OUT MORE...

Tender Process