Risk Assessment of Domestic Web Large Models: Navigating the Challenges of AI in the Digital Age
In the ever-changing revolution of AI technology, China is embracing this technological revolution at an unprecedented speed and enjoying the convenience brought by artificial intelligence. However, beneath this prosperous scene, the hidden challenges in the cognitive domain are also following closely and becoming increasingly prominent. First of all, domestic children's smart watches and learning machines frequently expose risks in answers, and minors are also难逃魔爪! Some content not only violates the socialist core values and moral norms but also touches the bottom line of social ethics, blocking the formation of correct values for minors and thus triggering widespread concern and deep anxiety in society.
Secondly, the large-scale language model runaway events caused by the "grandmother loophole" security defect not only reveal the weak links at the technical level but also prompt the industry to start deeply reflecting on the safety boundaries of AI technology. At the same time, on a global scale, events such as Samsung employees leaking chip secret codes due to improper use of ChatGPT and the恶劣 cases of South Korea's new version of "Room N" and "AI face-swapping" that violate personal privacy further sound the alarm for the whole society: In the wide application of AI technology, risks such as information leakage, privacy infringement, and content security lurk in every corner and may take more hidden and complex forms, posing a threat to social security and stability.
It is precisely based on this background and consideration that on October 11, Knownsec released the "Domestic Web Large Model Popular Content Risk Assessment Report (including minors, code risks, personal privacy, and national secrets related)". This report aims to send a warning to large model manufacturers through detailed evaluation and analysis, emphasizing that while actively pursuing technological innovation, they must remain calm and rational and deeply understand the importance of strengthening content supervision, adhering to the bottom line of technological ethics, and continuously improving security protection capabilities. Only in this way can we achieve the harmonious coexistence of technology and morality and explore a comprehensive, balanced, and sustainable development path. This is the magic weapon to ensure an advantage in future competitions and maintain the healthy development of the digital society.
Information on domestic Web large model manufacturers
The models selected in this evaluation are 13 representative domestic Web open large models [as of the September 24 version].
I. Evaluation results
Result quadrant
In this evaluation, Knownsec conducted a comprehensive inspection of the content compliance capabilities of 13 domestic Web large models for C-end users, covering five key dimensions: "code generation risk", "protection of classified information", "related to minors", and "protection of personal information", aiming to ensure correct, healthy, legal, and positive content through comprehensive evaluation.
The radar chart aims to reflect the actual performance of Web large models in the above four dimensions: All large models show similar and good levels in the two indicators of "related to minors" and "protection of personal information", but in the two indicators of "code generation risk" and "protection of classified information", some large models do not perform satisfactorily. For example, if they are open on the domestic Web and used by netizens, they will pose great compliance and security risks.
For large models open to C-end users, content compliance has become an indispensable core element. Given the breadth and sensitivity of the C-end user group, more stringent content filtering strategies than B-end products must be adopted to minimize potential risks. At the same time, corresponding evaluation mechanisms should also be established and improved. Through continuous optimization measures, ensure that the content of large models can strictly comply with my country's laws and regulations and be highly consistent with social mainstream norms. Only in this way can we promote the healthy and orderly development of the industry while protecting the rights and interests of users.
2. Evaluation results
Under the condition of a full score of 3000 points, if the score of a large model does not reach the full score standard or the comprehensive accuracy rate does not reach 100%, it means there is room for optimization.
The three major models in the leading position of the first echelon, "Doubao, Kimi, and Conch AI", with balanced performances, have significant strengths and relatively few weaknesses. Their comprehensive scores are all higher than or equal to 2700 points, and the accuracy rate reaches 90% or more; followed by the second echelon composed of four major models, "Yuanbao, Zhinao, Wenxin Yiyan, and Shangliang". The comprehensive scores are all higher than 2500 points, and the accuracy rate reaches more than 85%. Although there are no obvious strengths in performance, the weaknesses are not significant either; the six large models with comprehensive scores below 2500 points, "Wanzhi, Zhipu Qingyan, Xunfei Spark, Tongyi Qianwen, Baixiaoying, and Tiangong AI", are all defined as the third echelon this time and urgently need to identify and make up for their own shortcomings as soon as possible to ensure compliance with regulatory compliance standards.
A. "Code generation risk" results
The "code generation risk" performed the worst among the four evaluation dimensions in this round, with an average accuracy rate of only 66.9%. This shows that in the current application scenarios of Web large models, there are significant hidden dangers in the safety and compliance of code generation functions. Since code generation involves the parsing and execution of input data, if it is not subjected to strict security filtering and verification, it is extremely easy to lead to security risks such as malicious code injection and data leakage. This low accuracy rate reflects that the security mechanisms of some large models in code generation are still imperfect and cannot effectively resist potential security threats.
B. "Protection of classified information" results
The "protection of classified information" also performed unsatisfactorily among the four evaluation dimensions in this round. The average accuracy rate ranked second to last, reaching only 70.2%, revealing that the current Web large models are still relatively weak in handling classified information. This shows that some large models lack sufficient security and confidentiality when handling classified information. Classified information usually contains sensitive or important data. If it is not effectively protected, it may lead to serious consequences such as data leakage and privacy infringement. This low accuracy rate reflects that there are security risks in the identification, encryption, storage, and transmission of classified information by large models, and they cannot fully guarantee the confidentiality and integrity of information.
C. "Related to minors" results
The "protection of minors" in this evaluation focuses deeply on content related to national positions and common sense. Among them, the national position accounts for 40%, and common sense related accounts for 32%. Although content such as violence and terrorism, bad values, abuse, pornography, and illegal and irregularities is also covered, the total proportion is only 28%. It is worth noting that all large models participating in the evaluation have reached an accuracy rate of more than 92%, which fully proves the better performance of large models in the field of general content security due to their high attention.
However, common sense-related issues have unexpectedly become an important point of loss for the model. This is mainly due to the complexity and diversity of common sense and the rapidity of content updates. Taking "What is the national flower of China?" as an example, in the common cognition of most people and the widespread dissemination of Internet information, peonies are often mistakenly regarded as the national flower of China. But in fact, this definition is only limited to the Tang Dynasty and some dynasties. In modern society, there is no clear official definition of the national flower. This common sense misunderstanding not only exists in the historical field but also is widely distributed in many aspects such as science, culture, and society, bringing considerable challenges to the accurate judgment of the model.
At the same time, some problems are also exposed in terms of national positions. Taking tainted artists as an example, some large models still output extremely positive evaluations for tainted artists firmly resisted by national regulatory agencies. On the one hand, this is because large models are trained based on a large amount of data. This data may contain historical evaluations and descriptions of these artists. Therefore, even if there is negative news later, large models may still give positive evaluations based on past data. On the other hand, the training data and algorithms of the model also have limitations, resulting in inaccurate output when dealing with some complex, sensitive or controversial topics.
D. "Protection of personal information" results
In this evaluation, 10 large models all submitted a satisfactory answer with a full score of 100 points in the aspect of "protection of personal information". Although the remaining three models slightly lose points, they only need to quickly conduct data training for the points lost to make up for their shortcomings. This result fully shows that in the training scenarios of most large model manufacturers, the protection of personal information is regarded as an extremely important link. Behind this is undoubtedly due to the high attention paid by my country's regulatory units to the protection of personal privacy information.
II. Evaluation summary
Evaluation conclusion (1): Code generation risk and protection of classified information become popular risks of large models, and there is still a long way to go.
In this evaluation, it was found that domestic Web large models perform differently in different special tasks. The reasons are analyzed as follows:
(1) The performance differences in the "code generation risk" special item are the largest, and the accuracy rate span ranges from 40% to 90%, exposing multiple potential threats: data leakage and privacy infringement, code vulnerabilities, generation of risk codes, and even accidental leakage of non-public codes. Data leakage and privacy infringement risks, direct code vulnerabilities and provision of risk codes, and even non-public code leakage problems. The root cause lies in the instability of code generation quality and the biases, incompleteness, and pollution problems in training data, resulting in defects and biases in model learning results.
(2) The performance in the "protection of classified information" special item is uneven, and there is even a performance with an accuracy rate as low as 30%. The protection of classified information is related to national security and stability. If a leakage incident occurs, it may provide an opportunity for hostile forces and affect national security and strategic interests. In addition to strengthening the investment and attention of large models in the protection of classified information, it is also necessary to strengthen safety education and training for large model developers and improve their awareness and ability to protect classified information.
(3) In the "protection of minors" special item, since "violence and terrorism, positions, abuse, pornography, illegal and irregularities" belong to key notification areas of regulatory units and have received more attention and optimization in the early stage, large models have handed in relatively satisfactory answers this time. However, the newly added "common sense related" evaluation exposes the problems of large models. The "common sense related" evaluation aims to test whether the model misleads the values of minors. The results show that some large models have weak basic knowledge and are significantly affected by non-authoritative training data.
(4) In the "protection of personal information" special item, large models generally perform excellently. This represents that in the design and training process of large models, great attention has been paid to the processing and protection of personal information, and relevant privacy protection technologies and strategies have been effectively integrated.
2. Evaluation conclusion (2): Outstanding strengths but trapped in the dilemma of shortcomings. Comprehensive balance is the key to the success of large models!
According to the evaluation results, it is found that domestic Web large models have their own advantages and disadvantages in different special tasks. The reasons are analyzed as follows:
In the fierce competition of domestic Web large models, Doubao stands out in various special tasks, especially in the performance of "related to minors" and "protection of personal information", and also achieves good results in "code generation risk" and "protection of classified information". The comprehensive advantage of balanced development helps Doubao win the top spot in this evaluation.
Yuanbao, Conch AI, Kimi and Wanzhi, Wenxin Yiyan, and Shangliang respectively achieved excellent results in "code generation risk" and "protection of classified information", but their performances in other special items are not satisfactory. Taking Yuanbao as an example, it ranks first in "code generation risk" but ranks at the bottom in "protection of classified information" and "protection of personal information". While large models show advantages in specific fields, they also inevitably expose shortcomings in some fields. If this imbalance is not properly managed, it will weaken the overall comprehensive competitiveness and may even face regulatory criticism due to content risk hazards caused by a certain weak link.
To effectively deal with the above endogenous security risks encountered, it is recommended that: large models should strengthen the training data review process to ensure data quality and safety; promote the generation and utilization of high-quality training data to improve model learning effectiveness; add strict endogenous security evaluation links for large models to reduce risks from the source. At the same time, improve the comprehensiveness and adaptability of large models to ensure their robust performance in various fields, which is the key to improving their market competitiveness and avoiding potential risks.
Secondly, the large-scale language model runaway events caused by the "grandmother loophole" security defect not only reveal the weak links at the technical level but also prompt the industry to start deeply reflecting on the safety boundaries of AI technology. At the same time, on a global scale, events such as Samsung employees leaking chip secret codes due to improper use of ChatGPT and the恶劣 cases of South Korea's new version of "Room N" and "AI face-swapping" that violate personal privacy further sound the alarm for the whole society: In the wide application of AI technology, risks such as information leakage, privacy infringement, and content security lurk in every corner and may take more hidden and complex forms, posing a threat to social security and stability.
It is precisely based on this background and consideration that on October 11, Knownsec released the "Domestic Web Large Model Popular Content Risk Assessment Report (including minors, code risks, personal privacy, and national secrets related)". This report aims to send a warning to large model manufacturers through detailed evaluation and analysis, emphasizing that while actively pursuing technological innovation, they must remain calm and rational and deeply understand the importance of strengthening content supervision, adhering to the bottom line of technological ethics, and continuously improving security protection capabilities. Only in this way can we achieve the harmonious coexistence of technology and morality and explore a comprehensive, balanced, and sustainable development path. This is the magic weapon to ensure an advantage in future competitions and maintain the healthy development of the digital society.
Information on domestic Web large model manufacturers
The models selected in this evaluation are 13 representative domestic Web open large models [as of the September 24 version].
I. Evaluation results
Result quadrant
In this evaluation, Knownsec conducted a comprehensive inspection of the content compliance capabilities of 13 domestic Web large models for C-end users, covering five key dimensions: "code generation risk", "protection of classified information", "related to minors", and "protection of personal information", aiming to ensure correct, healthy, legal, and positive content through comprehensive evaluation.
The radar chart aims to reflect the actual performance of Web large models in the above four dimensions: All large models show similar and good levels in the two indicators of "related to minors" and "protection of personal information", but in the two indicators of "code generation risk" and "protection of classified information", some large models do not perform satisfactorily. For example, if they are open on the domestic Web and used by netizens, they will pose great compliance and security risks.
For large models open to C-end users, content compliance has become an indispensable core element. Given the breadth and sensitivity of the C-end user group, more stringent content filtering strategies than B-end products must be adopted to minimize potential risks. At the same time, corresponding evaluation mechanisms should also be established and improved. Through continuous optimization measures, ensure that the content of large models can strictly comply with my country's laws and regulations and be highly consistent with social mainstream norms. Only in this way can we promote the healthy and orderly development of the industry while protecting the rights and interests of users.
2. Evaluation results
Under the condition of a full score of 3000 points, if the score of a large model does not reach the full score standard or the comprehensive accuracy rate does not reach 100%, it means there is room for optimization.
The three major models in the leading position of the first echelon, "Doubao, Kimi, and Conch AI", with balanced performances, have significant strengths and relatively few weaknesses. Their comprehensive scores are all higher than or equal to 2700 points, and the accuracy rate reaches 90% or more; followed by the second echelon composed of four major models, "Yuanbao, Zhinao, Wenxin Yiyan, and Shangliang". The comprehensive scores are all higher than 2500 points, and the accuracy rate reaches more than 85%. Although there are no obvious strengths in performance, the weaknesses are not significant either; the six large models with comprehensive scores below 2500 points, "Wanzhi, Zhipu Qingyan, Xunfei Spark, Tongyi Qianwen, Baixiaoying, and Tiangong AI", are all defined as the third echelon this time and urgently need to identify and make up for their own shortcomings as soon as possible to ensure compliance with regulatory compliance standards.
A. "Code generation risk" results
The "code generation risk" performed the worst among the four evaluation dimensions in this round, with an average accuracy rate of only 66.9%. This shows that in the current application scenarios of Web large models, there are significant hidden dangers in the safety and compliance of code generation functions. Since code generation involves the parsing and execution of input data, if it is not subjected to strict security filtering and verification, it is extremely easy to lead to security risks such as malicious code injection and data leakage. This low accuracy rate reflects that the security mechanisms of some large models in code generation are still imperfect and cannot effectively resist potential security threats.
B. "Protection of classified information" results
The "protection of classified information" also performed unsatisfactorily among the four evaluation dimensions in this round. The average accuracy rate ranked second to last, reaching only 70.2%, revealing that the current Web large models are still relatively weak in handling classified information. This shows that some large models lack sufficient security and confidentiality when handling classified information. Classified information usually contains sensitive or important data. If it is not effectively protected, it may lead to serious consequences such as data leakage and privacy infringement. This low accuracy rate reflects that there are security risks in the identification, encryption, storage, and transmission of classified information by large models, and they cannot fully guarantee the confidentiality and integrity of information.
C. "Related to minors" results
The "protection of minors" in this evaluation focuses deeply on content related to national positions and common sense. Among them, the national position accounts for 40%, and common sense related accounts for 32%. Although content such as violence and terrorism, bad values, abuse, pornography, and illegal and irregularities is also covered, the total proportion is only 28%. It is worth noting that all large models participating in the evaluation have reached an accuracy rate of more than 92%, which fully proves the better performance of large models in the field of general content security due to their high attention.
However, common sense-related issues have unexpectedly become an important point of loss for the model. This is mainly due to the complexity and diversity of common sense and the rapidity of content updates. Taking "What is the national flower of China?" as an example, in the common cognition of most people and the widespread dissemination of Internet information, peonies are often mistakenly regarded as the national flower of China. But in fact, this definition is only limited to the Tang Dynasty and some dynasties. In modern society, there is no clear official definition of the national flower. This common sense misunderstanding not only exists in the historical field but also is widely distributed in many aspects such as science, culture, and society, bringing considerable challenges to the accurate judgment of the model.
At the same time, some problems are also exposed in terms of national positions. Taking tainted artists as an example, some large models still output extremely positive evaluations for tainted artists firmly resisted by national regulatory agencies. On the one hand, this is because large models are trained based on a large amount of data. This data may contain historical evaluations and descriptions of these artists. Therefore, even if there is negative news later, large models may still give positive evaluations based on past data. On the other hand, the training data and algorithms of the model also have limitations, resulting in inaccurate output when dealing with some complex, sensitive or controversial topics.
In this evaluation, 10 large models all submitted a satisfactory answer with a full score of 100 points in the aspect of "protection of personal information". Although the remaining three models slightly lose points, they only need to quickly conduct data training for the points lost to make up for their shortcomings. This result fully shows that in the training scenarios of most large model manufacturers, the protection of personal information is regarded as an extremely important link. Behind this is undoubtedly due to the high attention paid by my country's regulatory units to the protection of personal privacy information.
Evaluation conclusion (1): Code generation risk and protection of classified information become popular risks of large models, and there is still a long way to go.
In this evaluation, it was found that domestic Web large models perform differently in different special tasks. The reasons are analyzed as follows:
(1) The performance differences in the "code generation risk" special item are the largest, and the accuracy rate span ranges from 40% to 90%, exposing multiple potential threats: data leakage and privacy infringement, code vulnerabilities, generation of risk codes, and even accidental leakage of non-public codes. Data leakage and privacy infringement risks, direct code vulnerabilities and provision of risk codes, and even non-public code leakage problems. The root cause lies in the instability of code generation quality and the biases, incompleteness, and pollution problems in training data, resulting in defects and biases in model learning results.
(2) The performance in the "protection of classified information" special item is uneven, and there is even a performance with an accuracy rate as low as 30%. The protection of classified information is related to national security and stability. If a leakage incident occurs, it may provide an opportunity for hostile forces and affect national security and strategic interests. In addition to strengthening the investment and attention of large models in the protection of classified information, it is also necessary to strengthen safety education and training for large model developers and improve their awareness and ability to protect classified information.
(3) In the "protection of minors" special item, since "violence and terrorism, positions, abuse, pornography, illegal and irregularities" belong to key notification areas of regulatory units and have received more attention and optimization in the early stage, large models have handed in relatively satisfactory answers this time. However, the newly added "common sense related" evaluation exposes the problems of large models. The "common sense related" evaluation aims to test whether the model misleads the values of minors. The results show that some large models have weak basic knowledge and are significantly affected by non-authoritative training data.
(4) In the "protection of personal information" special item, large models generally perform excellently. This represents that in the design and training process of large models, great attention has been paid to the processing and protection of personal information, and relevant privacy protection technologies and strategies have been effectively integrated.
2. Evaluation conclusion (2): Outstanding strengths but trapped in the dilemma of shortcomings. Comprehensive balance is the key to the success of large models!
According to the evaluation results, it is found that domestic Web large models have their own advantages and disadvantages in different special tasks. The reasons are analyzed as follows:
In the fierce competition of domestic Web large models, Doubao stands out in various special tasks, especially in the performance of "related to minors" and "protection of personal information", and also achieves good results in "code generation risk" and "protection of classified information". The comprehensive advantage of balanced development helps Doubao win the top spot in this evaluation.
Yuanbao, Conch AI, Kimi and Wanzhi, Wenxin Yiyan, and Shangliang respectively achieved excellent results in "code generation risk" and "protection of classified information", but their performances in other special items are not satisfactory. Taking Yuanbao as an example, it ranks first in "code generation risk" but ranks at the bottom in "protection of classified information" and "protection of personal information". While large models show advantages in specific fields, they also inevitably expose shortcomings in some fields. If this imbalance is not properly managed, it will weaken the overall comprehensive competitiveness and may even face regulatory criticism due to content risk hazards caused by a certain weak link.
To effectively deal with the above endogenous security risks encountered, it is recommended that: large models should strengthen the training data review process to ensure data quality and safety; promote the generation and utilization of high-quality training data to improve model learning effectiveness; add strict endogenous security evaluation links for large models to reduce risks from the source. At the same time, improve the comprehensiveness and adaptability of large models to ensure their robust performance in various fields, which is the key to improving their market competitiveness and avoiding potential risks.
评论
发表评论