As an event organizer, what if you could digitally capture what attendees are experiencing as the event happens instead of relying on traditional post-event surveys? What if you could get real-time insight into people’s emotions at any given time or location during the event? Ruisrock, the most popular summer music festival in Finland, thought of the possibilities and turned to CGI for help. Through advanced video analytics, CGI enabled Ruisrock to capture more data on its 2017 festival crowd than it ever imagined, driving better decisions for an improved concert experience.

Digitalizing customer feedback

The first challenge with traditional customer feedback is that it`s just feedback—opinions based on past experiences and current expressions of the mind that may not accurately reflect the actual experience. The second challenge is that you don`t get much feedback at all, and the feedback often reflects one of two extremes—the experience was incredibly satisfying or incredibly dissatisfying. A third challenge is that traditional feedback is too general. For a large event, like Ruisrock, you can’t assess which part of the event created the best experiences and which did not.

For these reasons, Ruisrock`s partner, Mellakka Helsinki, a leading marketing agency in Finland, contacted CGI. It wanted to go beyond traditional surveys and capture real-time data on concertgoers, including their emotional states throughout the festival.

CGI answered the call. Through advanced video analytics, including facial analytics and custom neural networks, we set out to provide Ruisrock with rich, real-time data, including sentiment and demographic data, in real time and across festival grounds. With this data, Ruisrock gained valuable insight to help transform the concertgoer experience.

Collaborating through service design

Ruisrock’s 2017 summer festival was set for July 7-9. Our planning started in early 2017, and the CGI team delivered the project after only three months of service design workshops, planning and programming.

During the workshops, the CGI team demonstrated the business case and value of the project, including its long-term and short-term vision and impact. CGI, Ruisrock and Mellakka Helsinki also collaborated to define how to best capture the emotions and demographics of concertgoers to meet Ruisrock’s goals of customer segmentation and sentiment analysis.

We planned to get the best concert coverage by equipping 10-12 groups of volunteer workers with handheld GoPro cameras in addition to placing IP cameras at each of the festival’s 32 entry gates and primary VIP area. Video of concertgoers would provide hundreds of thousands of data points. In addition, by using our custom neural networks, we planned to capture data on concertgoers’ demographics (age and gender), as well as facial analytic information, such as their visible expressions and emotions.

All of the data capture, processing and analysis were planned in alignment with the compliance requirements of the General Data Protection Regulation (GDPR). For example, GDPR prohibits the recording of images of people without their permission.

However, cameras captured data only, not images, and the data identified only anonymous demographic information, such as gender and age. In addition, all of the data was destroyed after it was processed and analyzed, as required by GDPR.

Through this technology, Ruisrock, for the first time, would assess concertgoers’ emotions at any given time or location, including at a specific stage or vendor shop, or when a specific band or song played.

Pre-event implementation

Following a proof-of-concept, the implementation plan was put in place, and the technical planning and programming began. CGI provided all of the necessary hardware, including a multi-tier PaaS (Platform as a Service) to host the system. The cloud-based platform supports on-premise scalability and distributed processing, while also providing cloud data storage, high reliability and advanced data analytics.

Once the system was built, the CGI team tested it prior to setting it up on festival grounds. Subsequent onsite installation and testing took the team only one day. The team also managed all other technical operations, including camera and data visualization administration. In addition, during the event, an operations team monitored all of the equipment and quickly responded to issues.

Post-event data analysis

Following the event, we leveraged partner Qlick’s business intelligence software to help analyze the data and build data visuals. Data captured from the gate and VIP cameras was processed using edge processing. The GoPro data was processed using our onpremise server. Once the data was processed, it was transferred to the cloud. Advanced analytics and associated data visualization were then performed in the cloud.

In analyzing the data, a key challenge was overcoming visual obstacles that skewed the data (e.g., sunglasses, makeup, wigs, etc.). For example, makeup could make a 16-year-old girl look like a 30-year-old woman. We used advanced artificial intelligence techniques to overcome these data biases.

The results

Overall, 310 million data points were collected for analysis during Ruisrock’s 2017 summer festival. The data was captured through the recording of 1.6 million facial expressions and from 300,000 GPS locations. With the data, Ruisrock analyzed customer segments and emotions, such as identifying the happiest customer segments, including the happiest ages for both men and women.

Event locations, such as the main gates, stages and vendor spots, as well as specific performances and other aspects that impressed concertgoers the most also could be assessed. For example, the insight confirmed that happiness was measured highest near the Sateenkaari stage, right next to the entrance. That’s where Ruisrock had placed a giant unicorn and other visual effects. Based on this finding, Ruisrock will put even more effort in the entrance experience in the coming years.

Through video analytics, Ruisrock has moved beyond traditional customer feedback, gaining access to rich digital data and insight that will drive its transformation and a better concert experience both now and in the years to come.

"We've had successful years previously, but we wanted to know what we could do even better. Of course, we have always done the traditional surveys, but never before have we had a chance to study peoples’ emotional states during the festival. The biggest realization of this survey is what a large scale of emotions can be experienced during the festival, and from that we can draw conclusions on how to affect people’s memories of Ruisrock by offering the right services at a right time." Annakaisa Anttila, Producer, Ruisrock