IETF 122 post-meeting survey

The results of the post-meeting survey are now available on a interactive dashboard. Thank you to all of you who responded to this survey as we use your views to continually adjust the meeting experience.

Analysis

We received 244 responses, 196 onsite and 48 remote. With 1424 registered participants, this gives the survey a maximum margin of error of +/- 5.71%.

The results for satisfaction questions include a mean and standard deviation using a five point scale scoring system of Very satisfied = 5, Satisfied = 4, Neither satisfied nor dissatisfied = 3, Dissatisfied = 2, Very dissatisfied = 1. While there’s no hard and fast rule, a mean of above 4.50 is sometimes considered excellent, 4.00 to 4.49 is good, 3.50 to 3.99 is acceptable and below 3.50 is either poor or very poor if below 3.00. The satisfaction score tables also include a top box, the total of satisfied and very satisfied, and a bottom box, the total of dissatisfied and very dissatisfied, both in percentages. Please note that a small number of questions are on a four point scale.

Question changes since the last survey

For this survey a new question about side meetings was added and new questions relating to the revised new participant program. The question on avoiding conflicts was dropped as no further changes are planned to improve this. The question about various aspects of the hotel was reintroduced as this meeting was in a hotel but with some different choices.

Actions taken following the last survey

For this meeting, we made the following changes, prompted by survey feedback:

• The side meeting booking process was revised, as the first step in improving the side meeting experience overall.
• The amount of rented furniture was increased to supplement the venue provided furniture and to provide additional seating areas in the distributed spaces
• Break options for food were reviewed in light of better data about dietary requirements
• Directional signage was increased to guide participants through the distributed space

Satisfaction

Overall satisfaction is 4.49, a good result. With some key exceptions, the satisfaction scores remain high.

The table below shows the satisfaction scores for the last six meetings, along with colour coded indicators for the five point scale above: excellent (🔵), good (🟢), acceptable (🟡), poor (🔴), very poor (⚫️)

High satisfaction scores

Overall

The overall satisfaction score for this meeting was at the top end of good. Given that onsite participation was only 54% of total participation, this cannot all be down to the excellent food and beverage service during breaks.

Remote participation

The questions here were adjusted to focus on the quality of the remote experience and the satisfaction scores here are very good. The IETF has invested significantly in improving this experience, both financially and in changing processes, and this has clearly paid off.

Hackathon

The satisfaction score for the Hackathon jumped into the excellent range for the first time. though the survey otherwise has little evidence as to what led to that.

New Participant Program

Each of the individual areas of the New Participant Program that were surveyed about received a good satisfaction score, with an excellent overall score. There are some further improvements that could be made, and some specific comments to consider, but overall the revised program was real success.

Areas for improvement

Lunch break timing

The lunch break at 11:30am has never been popular. For IETF 123 Madrid an attempt will be made to have this later in the day, which is hoped will improve satisfaction here.

Side meetings

Scores for side meetings are slowly improving as minor, incremental changes are made but improvement is still needed. This will be a topic for discussion between the IESG, IETF LLC and Secretariat, before the next meeting and further changes are likely to come from that.

Individual comments

The individual comments covered a number of key themes, most of which have already been covered above, leaving the following:

• IETF 127. There were several comments recommending that the IETF does not meet in the US for IETF 127. The IETF LLC board issued a request for feedback regarding this matter, soon after the meeting and will be considering this at its May retreat.
• Room temperature. The meeting rooms were far too cold and this was not dealt with well enough. I apologise for that - we will try to ensure this does not happen again.
• Inconvenience of location/timezone. There were a number of comments recommending changes to our meeting booking policy to avoid inconveniences of location, travel or timezone. Unfortunately it is never going to be possible to find a location or timezone that suits everyone.
• Social events. One commenter asked why we were not having social events given "It costs pennies overall". Social events generally cost from $100,000 to $150,000 even with tickets sales accounted for, and are paid for solely by the meeting host, not the IETF. For several meetings now, meeting hosts have not had the budget to pay for these. If the IETF were to pay then that would mean an additional $100-$200 per registration, not including the cost of the social event ticket.
• Other comments. Other comments will be passed on to the appropriate people to consider.

And finally

Thank you everyone who responded to this survey, your feedback is much appreciated.