Selection of consultants to undertake large infrastructure works – Part Two

In Part One, we discussed the first stage of the Quality and Cost Based Selection (QCBS) process that kicked off with an Expression of Interest (EoI). At the tail end of this screening process, eight firms were shortlisted and each issued a Request for Proposal (RFP). 60 days later, the short-listed consultants submitted their bids after attending a successful pre-bid meeting.

Part Two of this article profiles my experience with Jeremy, an intern co-opted to participate in the evaluation of the submitted proposals.

Jeremy:

Good morning, Cyrus.

Me:

Good morning, Jeremy? How may I help you?

Jeremy:

I am an intern in my second year at university. For the next six weeks, I will be participating in the selection process for consultants to undertake large works before I relocate to an ongoing construction project north of the capital.

Me:

Great!

We are in the process of selecting a consultant to undertake consultancy services for a water and sewerage system. A couple of months ago, we completed the first stage of the selection process and are about to embark on the second stage.

The evaluation team has gathered here for a meeting to define the grades of the rating system for scoring the technical proposals, according to the evaluation criteria in the RFP.

Jeremy:

What is an RFP and why is this meeting necessary?

Me:

RFP is an acronym. It stands for Request for Proposal.

A Request for Proposal (RFP) is a solicitation document that provides all the instructions and information necessary for the short-listed consultants to prepare their proposals. In this case, the content and format of the RFP required prior approval from the financier before the short-listed consultants could receive invitations to submit their proposals.

The RFP comprises a letter of invitation, detailed instructions to the consultants, technical and financial proposal forms, terms of reference for the assignment and standard forms of contract for consultancy services i.e. a lump sum as well as a time-based form of contract. You must acquaint yourself with the content in the RFP before we commence the evaluation exercise.

You must acquaint yourself with the content in the RFP before we commence the evaluation exercise

Defining a grading system before proposal submission helps prevent bias (perceived bias) during the evaluation process. Normally, there is subjectivity in understanding the requirements in the terms of reference and hence, varying opinion in the interpretation of the evaluation criteria. Regardless of the competence of the evaluators, pre-definition of the grading system prevents the process from potential distortion.

To prepare yourself for the evaluation exercise, I will send you an email with a record of the proceedings for the consultants’ pre-bid meeting (conducted prior to bid submission) as well as a record of clarifications issued prior to bid submission.

The evaluation criteria

Jeremy:

Thanks for the documentation. I have learnt a great deal. Could you please explain the evaluation criteria in the RFP, the score allocation and the grading system?

Me:

Our evaluation comprises three main criteria. They are: (a) specific experience of the consultant relevant to the assignment (b) adequacy of the proposed methodology and work plan in responding to the terms of reference and (c) qualifications and competence of key staff for the assignment. Out of a maximum of 100 points, each criterion has been allocated 10, 40 and 50 points respectively.

Qualifications and competence of key staff for the assignment – 50 points

In the terms of reference and instructions to consultants, 13 key staff are required for this assignment. Do you recall seeing such a list of experts in the RFP?

Jeremy:

Yes, I do.

Me:

The team leader has the highest contribution to the total score of 50. To evaluate each key staff, sub criteria have been set for this criterion. Each expert will be assessed in three areas:

• general qualifications– 10 points
• adequacy for the assignment and – 25 points
• relevant experience in the region- 15 points

Sub criteria (i), (ii) and (iii) were allocated 10, 25 and 15 points respectively. For each sub criteria, the key experts will be graded as poor (40%), satisfactory (70%), good (90%) or very good (100%). For our evaluation, we have set 40% as the lowest grading for the sub criteria.

Jeremy:

Why is a poor grading assigned 40%?

Me:

Assigning a grade of 0% means the consultant has not responded at all to the terms of reference under this criterion. In addition, if a grade of 0% is applied to the sub criteria, it can potentially lead to premature rejection of a proposal which is (may be) attractive in criterion a) specific experience of the consultant relevant to the assignment and b) adequacy of the proposed methodology and work plan in responding to the terms of reference.

Assigning a grade of 0% means the consultant has not responded at all to the terms of reference under this criterion. In addition, if a grade of 0% is applied to the sub criteria it can potentially lead to premature rejection of a proposal which is (may be) attractive in criterion a) specific experience of the consultant relevant to the assignment and b) adequacy of the proposed methodology and work plan in responding to the terms of reference

Adequacy of the proposed methodology and work plan in responding to the terms of reference – 40 points

This criterion comprised three sub criteria:

• technical approach and methodology–20 points
• work plan– 12 points
• organisation and staffing–8 points

Sub criteria (i), (ii) and (iii) were allocated 20, 12 and 8 points respectively. For each sub criteria, the consultants will be graded as poor (40%), satisfactory (70%), good (90%) or very good (100%). The same reasons for non-application of a 0% grading for a poor submission apply.

Specific experience of the consultant relevant to the assignment – 10 points

Sub criteria have not been set for this criterion. Instead, we will assess the consultants’ specific experience as a whole. The specific experience will be assessed as satisfactory (40%), good (70%) or very good (100%). The same reasons for non-application of a 0% grading for a poor submission apply.

The technical evaluation

The technical evaluation progressed with ease.

Jeremy:

Thank you for the prior explanations. I observed each evaluator complete their assessment of the technical proposal for the shortlisted consultants. The excel sheets you sent by email were very handy.

For each consultant, the total score for each of the three evaluation criterion is derived from computing the product of the grading and the maximum possible points assigned to each sub criteria.

For each consultant, the total score for each of the three evaluation criterion is derived from  computing the product of the grading and the maximum possible points assigned to each sub criteria

Me:

That is correct. Have a look at this table.

For the shortlisted consultants, a summary of the total technical scores (a summation of scores from criterion a, b and c) for each consultant will be inserted in our report.

Jeremy:

I notice the technical evaluation counts for 75% of the overall assessment of the proposal. Does this mean that the financial evaluation counts for 25%?

Me:

In this case, that is correct. However, the technical evaluation could also account for 80% or 70% and the financial proposal 20% or 30%. By the way, company B, F and G have not met our minimum pass mark of 75% and will not to proceed to the next stage of the evaluation.

Company B, F and G have not met our minimum pass mark of 75% and will not to proceed to the next stage of the evaluation

Jeremy:

This part of the evaluation is complete. What shall we do next?

Me:

The financial proposals have not yet been opened.

We will proceed with evaluation of the financial proposals after we have received a No Objection to the outcome of the technical evaluation from the financier. To achieve that, we need to complete the bid evaluation report and submit our recommendation to the financier.

Jeremy:

Okay, but I still have some unanswered questions. I am hoping you are in position to answer them.

Me:

Hopefully.

Jeremy:

In awarding grades to sub criteria, it is normal to have discrepancies because certain evaluators are generous while others are rigid. From what I have observed, this has an effect on the scoring of the different criterion. How does the evaluation team manage such distortions?

Me:

It can be very difficult to explain and reconcile large differences resulting from varying interpretations of the criteria or sub criteria.

It can be very difficult to explain and reconcile large differences resulting from varying interpretations of the criteria or sub criteria

We convened a team meeting prior to commencement of the evaluation process to agree on the grading system. However, regardless of that, when all the technical scores were tallied, evaluator scores with a significant variation from the rest of the scores were investigated further to ensure consistency in the scoring of the different evaluators.

Jeremy:

You mentioned consultants can request clarifications to the RFP. By when must these clarifications be received? Surely, there must be a deadline!

Me:

Consultants may request clarifications to the RFP up to a certain number of days (indicated in the instructions to consultants) before the deadline of bid submission.

Jeremy:

Thank you. Please let me know when the financier has approved your evaluation report so that I can participate in stage three of the selection process.

Me:

Okay. I will do that

End of Part Two

© The Builders’ Garage 2019. Permission to use this article or quotations from it is granted subject to appropriate credit being given to thebuildersgarage.com as the source.