Preliminary Conference Agenda

Introduction. Large Language Models (LLMs) have become increasingly significant in crisis information management due to their advanced natural language processing capabilities. This study aims to develop a comprehensive evaluation benchmark to assess the effectiveness of LLMs in generating crisis information.

Method. CIEeval, an evaluation dataset, was constructed through steps such as information extraction and prompt generation. CIEeval covers 26 types of crises across sub-domains including water disasters, environmental pollution, and others, comprising a total of 4.8k data entries.

Analysis. Eight LLMs applicable to the Chinese context were selected for evaluation based on multidimensional criteria. A combination of manual and machine scoring methods was utilized. This approach ensured a comprehensive understanding of each model's performance.

Results. The manual and machine scores showed a significant correlation. Under this scoring method, Claude 3.5 Sonnet performed the best, particularly excelling in complex scenarios like natural and accident disasters. In contrast, while scoring slightly lower overall, Chinese models like ERNIE 4.0 Turbo and iFlytek Spark V4.0 showed strong performance in specific crises.

Conclusion. The evaluation benchmark validates the best LLM for crisis information generation (Claude 3.5 Sonnet) and provides valuable insights for LLMs to optimize and apply LLM in crisis information.

Climate change debates have divided our society more than ever. Despite most scientists believing in anthropogenic climate change, a small group of people with scientific knowledge and reasoning are denying it. In this paper, we collect YouTube video comments’ data to study the content posted by climate change skeptical scientists and their impact on comment social networks. We apply natural language processing and social networks analyses to study those comments and networks. We find that denying scientists question the validity of anthropogenic climate change using objective terms such as ‘co2’, ‘history’, ‘data’, etc., while non-scientists rarely mention these terms, instead frequently using words like ‘money’, ‘truth’. Scientists-led social networks are also more structured with significant core users, while non-scientists-led networks have smaller and fragmented groups, indicating scientists-led discussions on climate change are more stable and consistent. Scientists who deny human-caused climate change cast greater influence on the climate change denying social networks. Their opinions using more scientific terms cause the networks to be more centralized and form more consistent patterns.

Introduction. This paper investigates the consequences of using voice commands on YouTube to access information by a marginalised community—small-scale marine fishermen in Bangladesh. While YouTube, as a free platform, provides many users with opportunities to achieve their goals, it proves inadequate for communities with low literacy and digital skills.

Method. The research adopts a qualitative approach to investigate the socio-economic and cultural consequences of YouTube usage. Drawing on ethnographic fieldwork, it employs focus groups and interviews to explore the unique challenges this marginalised community faces in accessing and utilising information through the platform.

Analysis. The data were analysed using reflexive thematic analysis. Initially, codes were generated by categorising the types of information accessed on YouTube and the perceived societal consequences of this information. The analysis then shifted to identifying common patterns within these codes, ultimately leading to the development of key themes.

Results. The findings indicate that YouTube's content often fails to align with the specific needs of small-scale marine fishermen, frequently exposing them to irrelevant or harmful information. This exposure amplifies their existing vulnerabilities, leading to economic losses driven by information overload, increased domestic violence linked to the consumption of adult content, and a gradual degradation of societal norms.

Conclusion(s). This study concludes that while YouTube is unsuitable for small-scale marine fishermen, their preference for audio-visual content highlights the potential for a customised digital platform. Further research should investigate shared community spaces that combine physical and digital information, aligning with their diverse needs and capabilities to empower them with essential information.