Leveraging AI for Language Analytics in Daily Planning Conversations: A Novel Approach to Enhance Project Success

Overview

Effective communication and coordination during daily planning meetings are crucial for successful project execution and fostering a robust safety culture in the construction industry. However, conventional project management approaches, rooted in what the Project Production Institute (PPI) identifies as Era 1 and Era 2 thinking, have significant gaps that prevent the satisfactory management and execution of today’s complex and dynamic capital projects. This paper presents a novel approach that aligns with PPI’s Era 3 thinking, viewing projects as production systems and applying operations science to optimize delivery.

By integrating cutting-edge technologies, including artificial intelligence (AI), machine learning, natural language processing (NLP), and mobile technology, we propose a method to capture, transcribe, and evaluate the effectiveness of daily planning conversations at scale. This approach addresses the gap in conventional project management by focusing on work execution and variability, key elements of Project Production Management (PPM).

The paper introduces the “Simple Seven” meta-topic framework, derived from industry best practices, for assessing the comprehensiveness and impact of these dialogues. This framework aligns closely with PPI’s “five levers” of project production systems: product design, process design, capacity, inventory, and variability. By assessing conversations across these meta-topics, construction firms can cultivate a comprehensive understanding of their project production system, proactively address potential risks, gain insights into leadership effectiveness, and safety culture deployment.

The paper also presents research involving a partnership with the University at Buffalo, where machine learning models were developed to classify conversations across the seven meta-topics with an impressive 82% accuracy compared to human expert ratings. This automated analysis enables organizations to implement what PPI would call “Project Production Control,” offering real-time insights into how work is being planned and executed.

Preliminary findings demonstrate the potential of this approach to drive data-driven strategies, enhance project success, and foster a culture of safety through effective communication and coordination on construction sites. By providing concrete data on how work is being planned and executed, this methodology addresses the limitations of conventional project management and moves towards a more comprehensive understanding of projects as production systems, as advocated by PPI’s Era 3 vision.

Keywords: Construction planning, pre-job conversations, AI, natural language processing, machine learning, Project Production Management, variability, work-in-process, Era 3 thinking

Please log in or join to download.

Presentation

Authors

Chuck Pettinger, PhD

FactorLab

Chuck Pettinger, Ph.D. is a leading expert in organizational culture, metrics, and performance improvement. With over three decades of experience, Chuck has helped companies around the world create processes to better use their metrics to reduce error, improve performance and eliminate injuries. He has accomplished this by evaluating gaps in culture, assessi ...

Full Paper

Introduction

The construction industry is at a critical juncture, facing persistent challenges in project delivery that impact shareholder value for asset owners, developers, operators, contractors, and suppliers. As the Project Production Institute outlines, project management has evolved through three distinct eras, with the current era calling for a fundamental shift in how we approach project delivery1,2,3.

Effective communication and coordination during daily planning meetings are crucial for successful project execution in the construction industry. These meetings, commonly known as Job Hazard Analysis (JHA), Job Safety Analysis (JSA), or Pre-Task Plan (PTP) meetings, provide a vital opportunity for crews to align on the scope of work, expectations, identify potential hazards, and discuss safety precautions before beginning each workday. However, conventional project management approaches, rooted in Eras 1 and 2, have significant gaps that prevent the satisfactory management and execution of today’s complex and dynamic capital projects1,2,3.

Studies reveal that many construction pre-job conversations lack substantive engagement, psychological safety, thorough hazard analysis, and the culture needed to maximize their impact. For instance, Olson7 found that only 25% of the surveyed crews attended pre-task plans, and for those that attended, further analysis revealed that the craft only spent 0–2% of the time talking. Our research shows that 40% of the nearly 500,000 PTP morning conversations have only one person talking. This lack of discussion represents a missed opportunity for aligning on scope, daily tasks, proactive risk identification, and crew engagement4,5,6,7.

This paper presents a novel approach that aligns with PPI’s Era 3 thinking, viewing projects as production systems and applying operations science to optimize delivery. By integrating cutting-edge technologies, including artificial intelligence (AI), machine learning, natural language processing (NLP), and mobile technology, we propose a method to capture, transcribe, translate, and evaluate the effectiveness of daily planning conversations. This approach addresses the gap in conventional project management by focusing on work execution and variability, key elements of Project Production Management (PPM).

Our methodology incorporates aspects of PPI’s 4-5-3 framework, particularly addressing the “design” phase in optimizing daily work planning, and leveraging the levers of process design and variability reduction*. By providing real-time insights into conversation quality, our approach serves as a form of Project Production Control, enabling rapid adjustments to improve the production system of daily work planning and execution.

This research offers a practical approach to enhancing project success, fostering a culture of safety, and promoting effective communication and coordination on construction sites. By viewing daily planning conversations through the lens of PPM, we aim to optimize this critical aspect of project delivery, contributing to the broader goal of improving project outcomes in the construction industry.

Limitations of Written Documentation

While pre-task planning and documentation is an industry-standard practice aimed at proactively identifying hazards, there are significant limitations in relying solely on the written documents produced during these meetings. This overreliance on documentation aligns with what the Project Production Institute (PPI) identifies as Era 2 thinking, which focuses on planning and prediction but overlooks the critical aspects of work execution and variability management.

Figure 1: Sample Written PTP

The quality of information captured in these written documents is often dubious at best. Many pre-task plans simply restate repetitive tasks and boilerplate hazards without substantive context or control measures tailored to the specific work activities. This approach fails to address what PPI refers to as the “gap” in conventional project management – the lack of focus on how work is actually executed and the impact of variability on project outcomes.

Furthermore, the static nature of written documentation fails to capture the dynamic conversations, questions, and engagement levels that are crucial for effective hazard identification and promotion of psychological safety8. This limitation aligns with PPI’s critique of conventional project management’s inability to account for the impact of variability and work-in-process (WIP) during project execution.

Our analysis of over one million pre-task plan documents reveals that the quality of written information varies widely, with examples ranging from comprehensive discussions to cursory copy-and-paste jobs (see Figure 1). This variability in documentation quality illustrates the need for a more robust approach to understanding and influencing project production system behavior, as advocated by PPI’s Era 3 thinking.

Ultimately, the true value of pre-task planning lies in the dialogue and collective engagement fostered during the actual conversations themselves. By solely examining written documentation, construction firms lack critical insights into the crew’s level of participation, planning, psychological safety, and specific hazard topics discussed. This aligns with PPI’s emphasis on the importance of understanding and managing the “five levers” of project production systems: product design, process design, capacity, inventory, and variability.

By shifting focus from static documentation to dynamic conversation analysis, we can better address what PPI refers to as the “four verbs” of project production: design, make, transport, and build. In particular, this approach enhances the “design” aspect of daily work planning and execution, potentially leading to improved outcomes across all phases of project delivery.

In conclusion, while well-crafted pre-task plans can be beneficial, over-reliance on these documents alone is inadequate for assessing the quality and effectiveness of pre-job conversations. A more comprehensive approach, aligned with Project Production Management principles, is needed to truly optimize project delivery and outcomes.

The Value of Recorded Conversations

Historically, organizations could not objectively capture and analyze the nuances of workplace conversations beyond subjective observations from supervision. This limitation aligns with what the PPI identifies as Era 1 and Era 2 thinking in project management, which focused primarily on labor productivity and planning but lacked tools for understanding and optimizing work execution.

Pioneering work by Alex Pentland introduced techniques for gathering detailed conversational data using specialized sociometric badges equipped with sensors that captured over 100 data points per minute, including speech patterns, body language, turn-taking, and more9. By analyzing these metrics within teams, Pentland uncovered valuable insights, such as the correlation between higher conversational engagement through practices like equal participation and improved productivity.

While the use of specialized sensors may seem impractical in the dynamic and transient nature of construction sites, researchers have explored more cost-effective and scalable approaches. Barry Nelson recently leveraged readily available mobile devices (phones and tablets) to capture and analyze over 5,000 pre-job planning conversations10. By training site leaders to consistently record their daily pre-task planning meetings using mobile devices, Nelson’s team could gather key conversation characteristics like engagement levels, expressions of care, question quality, and hazard identification through video and audio recordings.

Nelson’s analysis of these planning conversations across 74 construction projects, each scored on critical conversation characteristics, revealed a striking correlation between higher conversation quality scores and improved safety outcomes. Specifically, projects with average conversation scores below 2.5 (out of 3) experienced nearly four times more injuries than those with scores above 2.511. These findings demonstrate the predictive power of leveraging conversation metrics and the scalability of using ubiquitous mobile devices to capture critical communication components.

Daily planning conversations between frontline leaders and work crews offer invaluable visibility into leadership effectiveness and the overall health of an organization’s safety systems8. As organizational experts have consistently found, high-quality interactions positively correlate with engaged employees, robust safety cultures, and reduced incidents12. The quality of these conversations serves as a proxy for leadership’s impact on the frontlines, with leaders who facilitate thoughtful and engaging planning sessions often demonstrating other sound safety practices, such as hazard recognition, transparent communication, and genuine care for crew welfare13.

Additional research has reinforced the value of crew-level conversations as indicators of organizational health. For instance, Pandit14 found that frequent informal discussions critically enable effective safety information flow, with crews who regularly engage in open dialogue about hazards and concerns demonstrating higher mutual trust and collaborative risk management. Conversely, when crews fail to sufficiently communicate relevant safety hazards and controls, the likelihood of injuries increases.

By monitoring the quality of daily field conversations through the practical application of readily available mobile devices, construction executives gain an invaluable window into leadership effectiveness, safety culture deployment, and overall system resilience9,8. Combined with leading indicators such as training and safety audit performance, the analysis of planning dialogues helps pinpoint areas where leadership support and coaching are most urgently needed, enabling proactive interventions to enhance project success and worker safety. The value of these recorded conversations extends beyond safety outcomes. As PPI emphasizes, understanding and optimizing the project production system is key to improving overall project performance. Daily planning conversations offer visibility into leadership effectiveness and the overall health of an organization’s production system. They serve as a proxy for “Project Production Control,” providing real-time insights into how work is being planned and executed.

In conclusion, the analysis of recorded conversations represents a significant advancement in project management, aligning with Era 3 thinking. By providing concrete data on how work is actually being planned and executed, this approach helps address the limitations of conventional project management and moves towards a more comprehensive understanding of projects as production systems.

Assessing Conversation Quality

While it intuitively makes sense that better-performing projects would have better-performing leadership and higher-quality pre-job conversations, evaluating the effectiveness of these discussions at scale has posed significant challenges. This challenge aligns with a key limitation of conventional project management approaches – the inability to effectively manage work execution and variability.

Historically, organizations have relied on time-consuming and resource-intensive methods to gauge the quality of pre-job discussions. These approaches, often fail to provide a comprehensive understanding of the conversations’ true effectiveness across an entire operation. They also fall short in addressing the “gap” in conventional project management – the lack of focus on how work is actually executed and the impact of variability on project outcomes.

Recent innovations in mobile technology have enabled a more practical and scalable solution: the ability to capture pre-planning conversations in the field, in real-time, through video recordings. This approach aligns with Era 3 thinking, which views projects as production systems and applies operations science to optimize delivery. By leveraging this capability, leadership teams can gain direct access to a wealth of authentic planning discussions, providing insights into critical aspects of the project production system.

The use of artificial intelligence (AI) and natural language processing (NLP) to automate the analysis and evaluation of these conversations at scale represents a significant advancement in Project Production Management. This technology allows for the efficient classification of conversations based on dimensions such as scope, capacity, hazard analysis, and engagement levels – all of which relate to the “five levers” of project production systems: product design, process design, capacity, inventory, and variability.

Nelson’s use of NLP and deep neural networks to categorize pre-job conversations demonstrates the potential of AI to reliably evaluate the quality of these dialogues at scale. This approach provides a means of implementing Project Production Control, offering real-time insights into how work is being planned and executed.

Automated conversation analysis enables organizations to establish comprehensive baselines for conversation quality across their operations and track improvements over time. This aligns with PPI’s emphasis on understanding and influencing project production system behavior. Furthermore, AI-powered assessments can deliver real-time insights to site leaders as pre-job meetings occur, fostering opportunities for immediate coaching and feedback – a key aspect of optimizing the “design” verb in PPI’s framework of project production.

By combining the power of AI with human expertise, construction firms can unlock a deeper understanding of the factors contributing to effective pre-job conversations. This approach addresses what PPI identifies as a critical need in project management: the ability to systematically account for the impact of variability and work-in-process (WIP) during project execution.

In conclusion, the use of AI and NLP to assess conversation quality represents a significant step towards Era 3 thinking in project management. By providing concrete data on how work is being planned and executed, this approach helps address the limitations of conventional project management and moves towards a more comprehensive understanding of projects as production systems.

Defining quality through topic analysis

To effectively assess the quality of daily planning conversations, it is necessary to analyze the specific topics and content discussed during these dialogues. By categorizing the sentences through NLP models and evaluating the nature of the topics addressed, construction firms can gain valuable insights into how the job is being planned, the expectations set by the foremen, the engagement levels of the craft, hazard awareness, and the overall effectiveness of these critical communication touchpoints.

Hazard Topic Models:

One fundamental aspect of high-quality pre-job safety conversations is the comprehensive identification and discussion of potential hazards relevant to the planned work activities. This aligns with PPI’s emphasis on understanding and influencing project production system behavior, particularly in relation to variability and risk management. Conversations that actively explore and discuss controlling hazards demonstrate a heightened level of risk awareness and a proactive approach to hazard planning, addressing key aspects of PPI’s “five levers” of project production systems, especially process design and variability reduction.

Figure 2: Sample Hazard Topic Model

Non-Hazard Topic Models:

While hazard identification is crucial, it’s equally important to recognize the value of addressing non-hazard topics that can significantly impact project success and worker well-being. Non-hazard topics such as capacity, setting expectations, and setting the scope of work serve as leading indicators of potential risks and provide valuable insights into the overall health and resilience of the project environment. These topics directly relate to PPI’s emphasis on capacity management and process design within the project production system.

Figure 3: Sample Non-Hazard Topic Model

Meta-Topic Analysis and Models:

In addition to analyzing individual hazard and non-hazard topics, it is crucial to evaluate the presence and quality of overarching meta-topics that encapsulate the essential elements of effective conversations. Drawing from extensive research and industry best practices, the “Simple Seven” meta-topics have emerged as a framework for assessing the comprehensiveness and impact of these morning conversations.

  1. Care: Do the conversations reflect trust? Is the dialogue creating or reducing capacity for unplanned events? Is the speaker demonstrating genuine interest in conversations or people? Is there any indication speakers are looking out for each other?
  2. Planning: Are expectations being set? Are they talking about the work or only hazards? Does everyone understand the expectations for the day? Do multiple people ask questions about the work as well as hazards?
  3. Hazards: What kinds of hazards are they talking about? Are they real ones? Who and how many people are talking about these hazards? How likely are they to report near incidents, unsafe conditions, or behaviors?
  4. Participant Engagement: Is this a monologue or dialogue? What kinds of questions are asked? How many people are participating? How much are they engaged? What level of collaboration is being seen?
  5. Observer Engagement: Measuring the leader’s engagement level during the conversation. Are they asking open-ended questions? Is the leader probing for understanding?
  6. Question Quality: Evaluating the quality of questions asked during the conversation. Is the only question being asked, “are there any questions”?
  7. High Hazards: Distinguishing between frequent hazards (e.g., not wearing PPE, slips, trips, and falls) and more consequential hazards or precursors often associated with potential serious injuries and fatalities (SIFs).

The “Simple Seven” meta-topics framework for assessing the comprehensiveness and impact of morning conversations aligns closely with PPI’s approach to Project Production Management. These meta-topics (Care, Planning, Hazards, Participant Engagement, Observer Engagement, Question Quality, and High Hazards) provide a structured way to evaluate critical aspects of the project production system, particularly related to process design, capacity management, and variability reduction.

By assessing conversations across these seven critical meta-topics, construction firms can cultivate a more comprehensive understanding of the project environment and proactively address potential risks. This provides real-time insights into how work is being planned and executed. The meta-topic scores serve as indicators of leadership effectiveness and project production system health, offering a concrete way to implement PPI’s vision of optimizing project delivery through a production system lens (see Figure 4 for example output).

Figure 4: the Simple 7 Meta-Topic Analysis

The conversation scoring system, ranging from 7-35, provides a quantifiable measure of conversation quality that can be applied at various organizational levels. This approach offers a practical way to understand project production system behavior across an entire organization. By focusing on what makes certain leaders or project cultures more effective in fostering high-impact morning planning conversations, organizations can work towards optimizing their entire project production system, as advocated by PPI’s Era 3 thinking.

In conclusion, this topic analysis approach represents a significant step towards implementing PPI’s vision of projects as production systems. By providing concrete data on how work is being planned and discussed, it addresses the limitations of conventional project management and moves towards a more comprehensive understanding of project dynamics, ultimately leading to improved project outcomes.

Figure 5: Example of an 8 Conversation Score
Figure 5: Example of an 8 Conversation Score
Figure 6: Example of a 15 Conversation Score
Figure 6: Example of a 15 Conversation Score
Figure 7: Example of a 31 Conversation Score
Figure 7: Example of a 31 Conversation Score

Practical Applications and Preliminary Findings – A Case Study

The strategies and recommendations outlined in this paper for enhancing daily pre-job safety conversations align closely with Era 3 thinking. Pioneering construction firms are already realizing substantial operational, and safety benefits by leveraging innovative AI technologies to activate their frontline workforce.

The following case study of a crane contractor provides a compelling example of successfully implementing AI-enabled analysis to optimize pre-job conversations. The contractor’s adoption of a mobile AI-powered platform for capturing and analyzing pre-job conversations represents a shift towards understanding and influencing project production system behavior.

The Safety Director’s goal of moving “attention away from backend, retrospective metrics to what was actively taking place in the field” aligns with a focus on real-time Project Production Control. This approach directly addresses the limitations of conventional project management by providing immediate insights into work execution and variability.

The platform’s real-time analytics and performance scoring of pre-job conversations provided unprecedented visibility into the company’s safety processes and engagement levels. The AI’s analysis of factors like scope, psychological safety, participation rates, and hazard analysis quality directly relates to PPI’s focus on process design and variability reduction.

The AI platform’s facilitation of more efficient yet substantive planning dialogues demonstrates the potential for technology to optimize a critical aspect of the project production system. The ability to conduct highly effective planning conversations in less than five minutes shows how AI can help address concerns about the efficient use of capacity in project delivery.

The tangible benefits achieved by the organization, including a 20% incident reduction in the first year and a 50% reduction in the second year, provide concrete evidence of the value of applying Era 3 thinking to project safety management.

This case study vividly demonstrates how innovative AI technologies can activate strategies for optimizing pre-job conversations within the broader context of Project Production Management. By arming leaders with objective, scalable insights into field realities, companies can strategically strengthen engagement skills, psychological safety, hazard analysis, accountability, and leadership commitment – all key components of an effective project production system.

In conclusion, while change management remains essential, this contractor’s success shows the immense potential of modern AI solutions to unlock the full potential of daily safety meetings as a catalyst for operational excellence. This aligns perfectly with PPI’s vision of Era 3 project management, where projects are optimized as production systems to deliver superior outcomes.

This paper presents a novel approach that harnesses the power of cutting-edge technologies to capture, transcribe, and analyze pre-job safety conversations at an unprecedented scale. By leveraging smartphone recordings, construction leaders can gain real-time visibility into the planning, engagement levels, and hazard analysis taking place during these dialogues.

The introduction of the “Simple Seven” meta-topic framework provides a comprehensive lens for evaluating the impact of these conversations, addressing key aspects of PPI’s “five levers” of project production systems: product design, process design, capacity, inventory, and variability.

By assessing factors such as psychological safety, hazard identification, leadership commitment, and question quality, construction firms can cultivate a deeper understanding of their project production system and proactively address potential risks.

Preliminary findings from pioneering firms demonstrate the tangible benefits of this approach, aligning with PPI’s assertion that applying operations science to project management can lead to significant improvements in outcomes. Contractors have achieved substantial reductions in incidents, enhanced operational efficiencies, and fostered a more engaged safety culture – all key indicators of an optimized project production system.

AI and NLP to Help Facilitate Era 3 Thinking

As the construction industry continues to evolve and embrace technological innovations, the integration of AI and language analytics into daily planning processes represents a significant step towards PPI’s Era 3 thinking. This approach holds immense promise for driving operational excellence, cultivating a robust safety culture, and ultimately saving lives by addressing what PPI identifies as the “gap” in conventional project management – the lack of focus on work execution and variability.

By empowering leaders with objective, scalable insights into field realities, this methodology enables targeted interventions, continuous improvement, and the strategic alignment of organizational values with frontline execution. This aligns perfectly with PPI’s emphasis on understanding and influencing project production system behavior to optimize outcomes.

While change management and leadership commitment remain essential, the success stories highlighted in this paper serve as a testament to the immense potential of modern AI solutions to unlock the full value of daily morning meetings. These solutions act as a catalyst for project success, worker well-being, and a thriving construction industry by implementing the principles of Project Production Management.

In conclusion, this AI-powered approach to optimizing daily planning conversations represents a significant advancement in project management, moving the industry towards the Era 3 vision. By providing concrete data on how work is being planned and executed, it addresses the limitations of conventional project management and moves towards a more comprehensive understanding of projects as production systems, ultimately leading to improved project outcomes across the construction industry.

References

  1. Shenoy, R., & Zabelle, T. (2016). New Era of Project Delivery – Project as Production System. https://projectproduction.org/journal/new-era-of-project-delivery-project-as-production-system/
  2. Zabelle, T., (2024). Built to Fail: Why Construction Projects Take So Long, Cost Too Much, and How to Fix It. Forbes Books.
  3. Zabelle, T., Choo, J., Spearman, M., & Pound, E. (2018). A “Gap” in Current Project Management and The Impact on Project Outcomes. https://projectproduction.org/journal/a-gap-in-current-project-management/
  4. Burke, M. J., Sarpy, S. A., Smith-Crowe, K., Chan-Serafin, S., Salvador, R. O., & Islam, G. (2006). Relative effectiveness of worker safety and health training methods. American Journal of Public Health, 96(2), 315–324. https://doi.org/10.2105/AJPH.2004.059840
  5. Edmondson, A. C. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44(2), 350-383. https://doi.org/10.2307/2666999
  6. Lingard, H., Wakefield, R., & Blismas, N. (2015). “If you cannot measure it, you cannot improve it”: Measuring health and safety performance in the construction industry. In Proc. 23rd Ann. Conf. of the Int’l. Group for Lean Construction. (pp. 29-31). Perth, Australia.
  7. Olson, R., Varga, A., Cannon, A., Jones, J., Gilbert-Jones, I., & Zoller, E. (2016). Toolbox talks to prevent construction fatalities: Empirical development and evaluation. Safety Science, 86, 122-131.
  8. Dekker, S. (2014). The field guide to understanding ‘human error’. CRC Press.
  9. Pentland, A. (2015). Social Physics: How Social Networks Can Make Us Smarter. Penguin press.
  10. Nelson, B. & Hirshfeld, S. (2021). Conversations are the new currency. FactorLab Whitepaper. https://factorlab.com/white-paper-conversations-are-the-new-currency/
  11. Nelson, B. (2020). What the Heck is a Proxy Metric and Why You Care. FactorLab Whitepaper. https://factorlab.com/what-the-heck-is-a-proxy-metric-and-why-you-care/
  12. Hallowell, M., Bhandari, S., & Alruqi, W. (2020). Methods of safety prediction: Analysis and integration of risk assessment, leading indicators, precursor analysis, and safety climate. Construction Management and Economics, 38(4), 308-321. https://doi.org/10.1080/01446193.2019.1598566
  13. Alsamadani, R., Hallowell, M., & Javernick-Will, A. N. (2013). Measuring and modelling safety communication in small work crews in the US using social network analysis. Construction Management and Economics, 31(6), 568-579.
  14. Pandit, B., Albert, A., Patil, Y., & Al-Bayati, A. J. (2018). Fostering safety communication among construction workers: Role of safety climate and crew-level cohesion. International Journal of Environmental Research and Public Health, 16(1), 71. https://www.mdpi.com/1660-4601/16/1/71/pdf
© 2024 Project Production Institute. All Rights Reserved.
X