Recently, the United StatesFederally fundedMITREThe company is currentlystayConduct creative artificial intelligence model testing based on the network battlefield scenario, aiming toTesting andDetermine the Large Language Model（LLM）Can network operations be enhanced or new security risks introduced in generating or identifying malicious code, especially in generating or identifying malicious code.

MITREThe company is a non-profit organization, formerly known as the Massachusetts Institute of TechnologyLincoln Laboratory Born during the Cold War era. The company primarily obtains research and development funding through funding from the US federal government and collaborations with other public and private enterprises or departments. Since its establishment,MITREWe have been committed to technological innovation and research and development, and have been repeatedly rated as the most suitable company to work for in the United States. We have achieved significant success in multiple fields, including but not limited to cybersecurity, public health, and defense construction.MITRE CorporationThis test is to understandLLMThe potential applications and limitations in defensive and offensive cybersecurity actions, as well as the significant impact they may have on the development and deployment of future artificial intelligence cybersecurity tools.

According to foreign media reports, this MITREutilizealreadyDevelop a comprehensive method for evaluating with customized toolsLLMAbility in a cybersecurity environment. mainadoptSimulate network combat scenariosTheSimulation of security reconnaissance toolsTheAttack scenario simulationMultiple testing scenarios for comprehensive evaluationLLMThe ability.These tests not only examineLLMThe mastery of basic knowledge and in-depth exploration of its decision-making ability in complex security reconnaissance and attack scenarios.

stayMITREIn the evaluation framework, targeting large language models（LLM）The ability test is designed in three progressive stages. firstyesSimulate network combat scenarios.In order to preliminarily investigate without any optimization or special adjustmentsLLMBased on the basic performance, the research team designed a set of multiple-choice questions around simulating network combat scenarios. These issues cover a wide range of topicsMITRE ATT&CKThe essence of the framework, through randomized question setting and option arrangement, effectively avoidsLLMThe possibility of answering based solely on memory ensures that the test can truly reflect their ability to understand and analyze complex network combat scenarios.

Subsequently, the evaluation entered the second phase of security reconnaissance tool simulation.Request thatLLMsimulationMITREFamous security reconnaissance tool“hound”. By simulating“hound”The workflow and functions of the tool,MITRECan directly observeLLMEvaluate the performance in handling security reconnaissance tasks and assess whether they possess advanced security analysis capabilities such as revealing hidden relationships and identifying attack paths，In depth explorationLLMThe potential in understanding and replicating complex security software.

lastyesAttack scenario simulation.In order to conduct a comprehensive inspectionLLMStrategic decision-making ability in offensive cybersecurity operations,MITREUtilizing advanced data generation modelsCyberLayerWe have built a highly dynamic and complex network environment.CyberLayerNot only did it create new network architectures, but it also constantly changedtopological structure And simulated complex interaction scenarios such as social networks. In such an environment,LLMThe task of simulating network attacks will rigorously test key decision-making abilities such as strategic planning, target selection, and attack execution.

The test results show that differentLLMThere are significant differences in the performance of the model in various tests, but overall it shows certain potential and limitations.Simulate network combat scenariostestinLarger onesLLMModels often perform better in handling complex problems, which may be related to their stronger semantic understanding and reasoning abilities.butLLMDealing with unknown situationsexistenceLimitations, future research should focus on how to improveLLMThe generalization ability.

Security reconnaissance toolstestDisplay,LLMThe ability to understand and replicate complex security software is closely related to its internal structure and training data. Future researchneedPay attention to how to optimizeLLMUse the structure and training data to enhance its performance.Attack SceneTests have shown that,LLMThe strategic decision-making ability in offensive scenarios is closely related to the size of its model and the quality of its training data. Larger onesLLMDue to its stronger semantic understanding and reasoning ability, the model has an advantage in formulating and executing attack strategies，Future research shouldMorePay attention to how to improveLLMAdaptability and flexibility.

MITREThe research on testing artificial intelligence models in cyber warfare scenarios is not only for understandingLLMThe potential in network security provides important insights and points the way for the development and deployment of future artificial intelligence network security tools.Currently, MITREWe are actively collaborating with a broader safety community to expand and improve our research methods.At the same time, they are also actively involvedSeeking more innovative testing ideas from security experts to collaboratively address the challenges posed by artificial intelligence in network security. Future research may focus on developing artificial intelligence defense tools and exploring ways to mitigate malicious use in network attacksLLMMethods for identifying potential risks.