Document Analysis
FDA Guidance Diff
Semantic document comparison that classifies regulatory changes. Validated against law firm analyses: 90-100% accuracy on modern FDA guidance documents.
122 changes detected
98 high impact
3 document pairs
Select Comparison
Total Changes
45
High Impact
43
Change Breakdown
NEW (33)
EXPANDED (3)
RESTRUCTURED (3)
REMOVED (6)
Filter
What do these change types mean?
NEW — Content only in newer document
REMOVED — Content only in older document
STRICTER — Requirements more stringent
MORE_LENIENT — Requirements less stringent
EXPANDED — Same topic, more detail added
CLARIFICATION — Same meaning, clearer language
RESTRUCTURED — Content moved/reorganized
EDITORIAL — Cosmetic changes only
EXPANDED
HIGH IMPACT
II. Scope
The scope of applicable devices and submission types has been significantly broadened to include more device characteristics, additional submission types, and devices that do not require a premarket submission.
Before (p. 4)
2. Scope This guidance provides recommendations to consider and information to include in FDA medical device premarket submissions for effective cybersecurity management. Effective cybersecurity management is intended to reduce the risk to patient
After (pp. 6-7)
II. Scope This guidance is applicable to devices with cybersecurity considerations, including but not limited to devices that include a device software function1 or that contain software (including firmware) or programmable logic. The guidance is
EXPANDED
HIGH IMPACT
1. Implementation of Security Controls
The new version significantly expands on the requirement for manufacturers to integrate cybersecurity into the design process, explicitly referencing specific CFR sections and detailing the need for design inputs, outputs, and acceptance criteria for security features.
Before (pp. 5-6)
4. General Principles Manufacturers should develop a set of cybersecurity controls to assure medical device cybersecurity and maintain medical device functionality and safety. FDA recognizes that medical device security is a shared responsibili
After (pp. 25-26)
1. Implementation of Security Controls FDA considers the way in which a device addresses cybersecurity risks and the way in which the device responds when exposed to cybersecurity threats as functions of the device design. Effective cybersecurity
EXPANDED
HIGH IMPACT
B. Designing for Security
The new version expands on the cybersecurity functions by defining specific security objectives (Authenticity, Authorization, Availability, Confidentiality, and Secure and timely updatability and patchability) that manufacturers must address in their device design, moving beyond the general 'Identify, Protect, Detect, Respond, and Recover' framework.
Before (pp. 6-8)
5. Cybersecurity Functions The Agency recommends that medical device manufacturers consider the following cybersecurity framework core functions to guide their cybersecurity activities: Identify, Protect, Detect, Respond, and Recover.5 Identify
After (p. 11)
B. Designing for Security When reviewing premarket submissions, FDA intends to assess device cybersecurity based on a number of factors, including, but not limited to, the device’s ability to provide and implement the security objectives below th
NEW
HIGH IMPACT
I. Introduction
New content: I. Introduction
No previous content
After (pp. 5-6)
I. Introduction With the increasing integration of wireless, Internet- and network-connected capabilities, portable media (e.g., USB or CD), and the frequent electronic exchange of medical device- related health information and other information,
NEW
HIGH IMPACT
III. Background
New content: III. Background
No previous content
After (pp. 7-9)
III. Background FDA recognizes that medical device cybersecurity is a shared responsibility among interested parties throughout the use environment of the medical device system, including healthcare facilities, patients, healthcare providers, and
NEW
HIGH IMPACT
IV. General Principles
New content: IV. General Principles
No previous content
After (p. 9)
IV. General Principles This section provides general principles for device cybersecurity relevant to device manufacturers. The principles in this guidance are important to the improvement of device cybersecurity and, when followed, are expected to
NEW
HIGH IMPACT
1. A Secure Product Development Framework (SPDF) may be one way to satisfy the
New content: 1. A Secure Product Development Framework (SPDF) may be one way to satisfy the
No previous content
After (p. 3)
1. A Secure Product Development Framework (SPDF) may be one way to satisfy the QS regulation 6 B. Designing for Security
NEW
HIGH IMPACT
1. A Secure Product Development Framework (SPDF) may be
New content: 1. A Secure Product Development Framework (SPDF) may be
No previous content
After (pp. 10-11)
1. A Secure Product Development Framework (SPDF) may be one way to satisfy the QS regulation Cybersecurity threats have the potential to exploit one or more vulnerabilities that could lead to patient harm. The greater the number of vulnerabilitie
NEW
HIGH IMPACT
C. Transparency
New content: C. Transparency
No previous content
After (pp. 11-12)
C. Transparency A lack of cybersecurity information, such as information necessary to integrate the device into the use environment, as well as information needed by users to maintain the medical device system’s cybersecurity over the device life
NEW
HIGH IMPACT
D. Submission Documentation
New content: D. Submission Documentation
No previous content
After (pp. 12-13)
D. Submission Documentation Device cybersecurity design and documentation are expected to scale with the cybersecurity risk of that device. Manufacturers should take into account the larger system in which the device may be used. For example, a c
NEW
HIGH IMPACT
V. Using an SPDF to Manage Cybersecurity Risks
New content: V. Using an SPDF to Manage Cybersecurity Risks
No previous content
After (pp. 13-14)
V. Using an SPDF to Manage Cybersecurity Risks The documentation recommended in this guidance is based on FDA’s experience evaluating the safety and effectiveness of devices with cybersecurity vulnerabilities. However, sponsors may use alternative
NEW
HIGH IMPACT
A. Security Risk Management
New content: A. Security Risk Management
No previous content
After (pp. 14-16)
A. Security Risk Management To fully account for cybersecurity risks in medical device systems, the safety and security risks of each device should be assessed within the context of the larger system in which the device operates. In the context o
NEW
HIGH IMPACT
1. Threat Modeling
New content: 1. Threat Modeling
No previous content
After (pp. 16-17)
1. Threat Modeling Threat modeling includes a process for identifying security objectives, risks, and vulnerabilities across the medical device system, and then defining countermeasures to prevent, mitigate, monitor, or respond to the effects of
NEW
HIGH IMPACT
2. Cybersecurity Risk Assessment
New content: 2. Cybersecurity Risk Assessment
No previous content
After (pp. 17-18)
2. Cybersecurity Risk Assessment As a part of security risk management, security risks and controls should be assessed for residual risks as part of a cybersecurity risk assessment. Effective security risk assessments address the fact that cybers
NEW
HIGH IMPACT
3. Interoperability Considerations
New content: 3. Interoperability Considerations
No previous content
After (pp. 18-19)
3. Interoperability Considerations Interoperability is an important consideration when assessing the cybersecurity of the end-to-end medical device system. As identified in FDA’s guidance “Design Considerations and Pre-market Submission Recommend
NEW
HIGH IMPACT
4. Third-Party Software Components
New content: 4. Third-Party Software Components
No previous content
After (pp. 19-22)
4. Third-Party Software Components As discussed in FDA’s guidance “Off-The-Shelf (OTS) Software Use in Medical Devices,” medical devices commonly include third-party software components,33 including off-the-shelf and open source software. When th
NEW
HIGH IMPACT
5. Security Assessment of Unresolved Anomalies
New content: 5. Security Assessment of Unresolved Anomalies
No previous content
After (p. 22)
5. Security Assessment of Unresolved Anomalies FDA’s Premarket Software Guidance recommends that device manufacturers provide a list of software anomalies that exist in a product at the time of submission. For each anomaly, FDA recommends that d
NEW
HIGH IMPACT
6. TPLC Security Risk Management
New content: 6. TPLC Security Risk Management
No previous content
After (pp. 22-23)
6. TPLC Security Risk Management Cybersecurity risks may continue to be identified throughout the device’s TPLC. Manufacturers should ensure they have appropriate resources to identify, assess, and mitigate cybersecurity vulnerabilities as they a
NEW
HIGH IMPACT
B. Security Architecture
New content: B. Security Architecture
No previous content
After (pp. 23-25)
B. Security Architecture Manufacturers are responsible for identifying cybersecurity risks in their devices and the systems in which they expect those devices to operate, and implementing the appropriate controls to mitigate those risks. These r
NEW
HIGH IMPACT
2. Security Architecture Views
New content: 2. Security Architecture Views
No previous content
After (pp. 26-29)
2. Security Architecture Views In addition to the design control requirements,44 21 CFR 820.100 requires that manufacturers establish and maintain procedures for implementing corrective and preventive action, which must include, among other thing
NEW
HIGH IMPACT
C. Cybersecurity Testing
New content: C. Cybersecurity Testing
No previous content
After (pp. 29-31)
C. Cybersecurity Testing As with other areas of product development, testing is used to demonstrate the effectiveness of design controls. While software development and cybersecurity are closely related disciplines, cybersecurity controls require
NEW
HIGH IMPACT
VI. Cybersecurity Transparency
New content: VI. Cybersecurity Transparency
No previous content
After (p. 31)
VI. Cybersecurity Transparency Cybersecurity transparency is critical to ensure safe and effective use and integration of devices and systems.48 This transparency can be conveyed through both device labeling and the establishment of manufacturer v
NEW
HIGH IMPACT
B. Cybersecurity Management Plans
New content: B. Cybersecurity Management Plans
No previous content
After (p. 34)
B. Cybersecurity Management Plans Recognizing that cybersecurity risks evolve as technology evolves throughout a device’s TPLC, FDA recommends that manufacturers establish a plan for how they will identify and communicate to the relevant parties
NEW
HIGH IMPACT
VII. Cyber Devices
New content: VII. Cyber Devices
No previous content
After (pp. 34-35)
VII. Cyber Devices This section identifies the cybersecurity information FDA considers to generally be necessary to support obligations under section 524B of the FD&C Act for cyber devices. This section provides recommendations specifically for cy
NEW
HIGH IMPACT
A. Who is Required to Comply with Section 524B of the
New content: A. Who is Required to Comply with Section 524B of the
No previous content
After (p. 35)
A. Who is Required to Comply with Section 524B of the FD&C Act Under section 524B(a) of the FD&C Act, a person, including a manufacturer,52 who submits a premarket application or submission under any of the following pathways—510(k),53 PMA,54 PDP
NEW
HIGH IMPACT
2. Design, Develop, and Maintain Processes and Procedures to Provide a Reasonable
New content: 2. Design, Develop, and Maintain Processes and Procedures to Provide a Reasonable
No previous content
After (pp. 3-5)
2. Design, Develop, and Maintain Processes and Procedures to Provide a Reasonable Assurance of Cybersecurity (Section 524B(b)(2)) 34 Contains Nonbinding Recommendations 3. Softw
NEW
HIGH IMPACT
C. Documentation Recommendations to Comply with
New content: C. Documentation Recommendations to Comply with
No previous content
After (p. 36)
C. Documentation Recommendations to Comply with Section 524B of the FD&C Act For applicable premarket submission types, manufacturers must provide documentation to comply with the requirements under section 524B of the FD&C Act. Recommendations r
NEW
HIGH IMPACT
1. Plans and Procedures (Section 524B(b)(1))
New content: 1. Plans and Procedures (Section 524B(b)(1))
No previous content
After (pp. 36-38)
1. Plans and Procedures (Section 524B(b)(1)) Section 524B(b)(1) of the FD&C Act requires manufacturers of cyber devices to submit to FDA “a plan to monitor, identify, and address, as appropriate, in a reasonable time, postmarket cybersecurity vuln
NEW
HIGH IMPACT
2. Design, Develop, and Maintain Processes and
New content: 2. Design, Develop, and Maintain Processes and
No previous content
After (pp. 38-39)
2. Design, Develop, and Maintain Processes and Procedures to Provide a Reasonable Assurance of Cybersecurity (Section 524B(b)(2)) Manufacturers of cyber devices must “design, develop, and maintain processes and procedures to provide a reasonable
NEW
HIGH IMPACT
3. Software Bill of Materials (SBOM) (Section 524B(b)(3))
New content: 3. Software Bill of Materials (SBOM) (Section 524B(b)(3))
No previous content
After (p. 39)
3. Software Bill of Materials (SBOM) (Section 524B(b)(3)) Section 524B(b)(3) of the FD&C Act requires manufacturers of cyber devices to provide an SBOM, including commercial, open-source, and off-the-shelf software components. To assist with compl
NEW
HIGH IMPACT
D. Modifications
New content: D. Modifications
No previous content
After (p. 39)
D. Modifications As previously stated, the requirements under section 524B of the FD&C Act apply to a manufacturer who submits an application or submission under any of the following pathways— 510(k), PMA, PDP, De Novo or HDE—for a device that meet
NEW
HIGH IMPACT
1. Changes That May Impact Cybersecurity
New content: 1. Changes That May Impact Cybersecurity
No previous content
After (p. 39)
1. Changes That May Impact Cybersecurity In general, changes that may impact cybersecurity and may require premarket submission could include changes to authentication or encryption algorithms, new connectivity features, or changing software upda
NEW
HIGH IMPACT
2. Changes Unlikely to Impact Cybersecurity
New content: 2. Changes Unlikely to Impact Cybersecurity
No previous content
After (pp. 39-40)
2. Changes Unlikely to Impact Cybersecurity In general, changes unlikely to impact cybersecurity could include changes in materials, sterilization method changes, or a change to an algorithm without change to architecture/software structure/connec
NEW
HIGH IMPACT
E. Reasonable Assurance of Cybersecurity of Cyber
New content: E. Reasonable Assurance of Cybersecurity of Cyber
No previous content
After (pp. 40-51)
E. Reasonable Assurance of Cybersecurity of Cyber Devices Section 3305(c) of FDORA provides that nothing in section 524B of the FD&C Act “shall be construed to affect the Secretary’s authority related to ensuring that there is a reasonable assura
NEW
HIGH IMPACT
A. Diagrams
New content: A. Diagrams
No previous content
After (p. 51)
A. Diagrams FDA recommends that manufacturers provide diagrams to help describe the medical device system architecture, interfaces, communication protocols, threats, and cybersecurity controls used throughout the system. Different diagramming met
NEW
HIGH IMPACT
B. Information Details for an Architecture View
New content: B. Information Details for an Architecture View
No previous content
After (pp. 51-64)
B. Information Details for an Architecture View For each view described in Section V.B.2, manufacturers should provide a system-level description and analysis inclusive of end-to-end security analyses of all the communications in the medical devi
REMOVED
HIGH IMPACT
1. Introduction
Removed: 1. Introduction
Before (pp. 3-4)
1. Introduction The need for effective cybersecurity to assure medical device functionality and safety has become more important with the increasing use of wireless, Internet- and network- connected devices, and the frequent electronic exchange o
Content removed
REMOVED
HIGH IMPACT
2. A traceability matrix that links your actual cybersecurity controls to the
Removed: 2. A traceability matrix that links your actual cybersecurity controls to the
Before (p. 8)
2. A traceability matrix that links your actual cybersecurity controls to the cybersecurity risks that were considered;
Content removed
REMOVED
HIGH IMPACT
3. A summary describing the plan for providing validated software updates and
Removed: 3. A summary describing the plan for providing validated software updates and
Before (p. 8)
3. A summary describing the plan for providing validated software updates and patches as needed throughout the lifecycle of the medical device to continue to assure its safety and effectiveness. The FDA typically will not need to review or approv
Content removed
REMOVED
HIGH IMPACT
5. Device instructions for use and product specifications related to recommended
Removed: 5. Device instructions for use and product specifications related to recommended
Before (p. 8)
5. Device instructions for use and product specifications related to recommended cybersecurity controls appropriate for the intended use environment (e.g. anti-virus software, use of firewall)
Content removed
REMOVED
HIGH IMPACT
7. Recognized Standards
Removed: 7. Recognized Standards
Before (pp. 8-9)
7. Recognized Standards The following is a list of FDA recognized consensus standards dealing with Information Technology (IT) and medical device security. Contains Nonbinding Recommendations 7
Content removed
REMOVED
HIGH IMPACT
1. CLSI, AUTO11-A - IT Security of In Vitro Diagnostic Instruments and
Removed: 1. CLSI, AUTO11-A - IT Security of In Vitro Diagnostic Instruments and
Before (p. 9)
1. CLSI, AUTO11-A - IT Security of In Vitro Diagnostic Instruments and Software Systems; Approved Standard. 2. IEC, TR 80001-2-2 Edition 1.0 2012-07 - Application of risk management for IT Networks incorporating medical devices - Part 2-2: Guid
Content removed
RESTRUCTURED
HIGH IMPACT
B. Devices Subject to Section 524B of the FD&C Act
The new version defines 'cyber device' based on the FD&C Act, replacing the previous general definitions of cybersecurity terms.
Before (pp. 4-5)
3. Definitions Asset2 - is anything that has value to an individual or an organization. Authentication - is the act of verifying the identity of a user, process, or device as a prerequisite to allowing access to the device, its data, information,
After (pp. 35-36)
B. Devices Subject to Section 524B of the FD&C Act Section 524B of the FD&C Act and its requirements apply to “cyber devices.” Section 524B(c) of the FD&C Act defines a “cyber device” as a device that meets all of the following criteria “(1) inc
RESTRUCTURED
A. Cybersecurity is Part of Device Safety and the Quality
The new version recontextualizes cybersecurity documentation within the broader Quality System Regulation and its relevance at different stages, rather than directly listing specific documentation requirements as in the old version.
Before (p. 8)
6. Cybersecurity Documentation The type of documentation the Agency recommends you submit in your premarket submission is summarized in this section. These recommendations are predicated on your effective implementation and management of a qual
After (pp. 9-10)
A. Cybersecurity is Part of Device Safety and the Quality System Regulation Device manufacturers must establish and follow quality systems to help ensure that their products consistently meet applicable requirements and specifications. The qualit
RESTRUCTURED
A. Labeling Recommendations for Devices with
The old version listed specific technical standards related to risk management and network security, while the new version references different standards in a footnote related to cybersecurity risks and labeling recommendations, indicating a shift in focus and organization rather than a direct update of the same list.
Before (p. 9)
3. AAMI/ANSI/IEC, TIR 80001-2-2:2012, - Application of risk management for IT Networks incorporating medical devices - Part 2-2: Guidance for the disclosure and communication of medical device security needs, risks and controls. 4. IEC, /TS 6244
After (pp. 31-34)
A. Labeling Recommendations for Devices with Cybersecurity Risks FDA regulates device labeling in several ways. For example, section 502(f) of the FD&C Act requires that labeling include adequate directions for use. Under section 502(a)(1) of the
Total Changes
28
High Impact
8
Change Breakdown
NEW (4)
EXPANDED (15)
CLARIFICATION (5)
EDITORIAL (1)
REMOVED (3)
Filter
What do these change types mean?
NEW — Content only in newer document
REMOVED — Content only in older document
STRICTER — Requirements more stringent
MORE_LENIENT — Requirements less stringent
EXPANDED — Same topic, more detail added
CLARIFICATION — Same meaning, clearer language
RESTRUCTURED — Content moved/reorganized
EDITORIAL — Cosmetic changes only
EXPANDED
HIGH IMPACT
A. Software Functions
The new version expands on the definition of software functions by introducing and defining 'Artificial Intelligence' and 'Machine Learning' as foundational concepts, and renames 'Machine Learning-Enabled Device Software Function' to 'Artificial Intelligence-Enabled Device Software Function' to reflect this broader scope.
Before (p. 12)
A. Software Functions 219 Device Software Function (DSF): A software function that meets the device definition in 220 section 201(h) of the FD&C Act.35,36 As discussed in other FDA guidances, the term “function” 221 is a distinct purpose of th
After (p. 12)
A. Software Functions Artificial Intelligence (AI): A machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments. Artificial intelligence sys
NEW
HIGH IMPACT
I. Introduction
New content: I. Introduction
NEW
HIGH IMPACT
V. Policy for Predetermined Change Control Plans
New content: V. Policy for Predetermined Change Control Plans
NEW
HIGH IMPACT
F. Version Control and Maintenance of a PCCP for a Device
New content: F. Version Control and Maintenance of a PCCP for a Device
No previous content
After (pp. 23-24)
F. Version Control and Maintenance of a PCCP for a Device At this time, as manufacturers gain experience developing and implementing PCCPs, FDA anticipates that review of a marketing submission that includes a PCCP will be interactive. It is poss
NEW
HIGH IMPACT
A. Goals of the Description of Modifications Section
New content: A. Goals of the Description of Modifications Section
No previous content
After (pp. 24-25)
A. Goals of the Description of Modifications Section A dedicated Description of Modifications section in a PCCP should identify the specific, planned modifications to the AI-DSF that the manufacturer intends to implement. The Description of Modif
REMOVED
HIGH IMPACT
I. Introduction
Removed: I. Introduction
REMOVED
HIGH IMPACT
V. Policy for Predetermined Change Control Plans
Removed: V. Policy for Predetermined Change Control Plans
REMOVED
HIGH IMPACT
A. Goals of the Description of Modifications Section
Removed: A. Goals of the Description of Modifications Section
Before (p. 20)
A. Goals of the Description of Modifications Section 448 A dedicated Description of Modifications section in a PCCP identifies the specific, planned 449 modifications to the ML-DSF that the manufacturer intends to implement. The Description of
Content removed
EXPANDED
II. Background
The new version significantly expands on the definitions provided in the footnotes, introducing new terms like 'AI-enabled device' and 'AI-DSF' and referencing additional guidance documents.
Before (pp. 6-9)
II. Background 55 In April 2019, FDA published the Proposed Regulatory Framework for Modifications to 56 Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) 57 2 For the purposes of this guidance, “manuf
After (pp. 6-9)
II. Background In April 2019, FDA published the “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) - Discussion Paper and Request for Feedback” (“2019 disc
EXPANDED
III. Scope
The new version expands on the scope by explicitly including a combination of automatic and manual modifications, and clarifies that while the guidance is broadly applicable to AI-DSFs, many recommendations are tailored to ML-DSFs due to FDA's review experience.
Before (pp. 9-12)
III. Scope 147 This draft guidance is applicable to ML-DSFs that the manufacturer intends to modify over time. 148 This includes ML-DSFs for which modifications to the ML model are implemented 149 automatically (i.e., for which the modification
After (pp. 9-12)
III. Scope This guidance is applicable to AI-DSFs that the manufacturer intends to modify over time. This includes AI-DSFs for which modifications to the AI model are implemented automatically (i.e., for which the modifications are implemented aut
EXPANDED
B. Data Sets
The new version adds a new section for 'Test Data' and expands on the representativeness of training data to include intended environments, while also clarifying the FDA's stance on the term 'validation' in the context of tuning data.
Before (pp. 12-13)
B. Data Sets 228 Training Data: These data are used by the ML-DSF manufacturer in procedures and ML 229 training algorithms to build an ML model, including to define model weights, connections, and 230 components. Training the ML model happens
After (pp. 12-13)
B. Data Sets Training Data: These data are used by the manufacturer of an AI-DSF in procedures and training algorithms to build an AI model, including to define model weights, connections, and 34 15 U.S.C. 9401(3). 35 15 U.S.C. 9401(11). 36 Se
EXPANDED
V. Policy for Predetermined Change Control Plans
The new version significantly expands on the structure and required content of an authorized Predetermined Change Control Plan (PCCP) by outlining three specific sections: Description of Modifications, Modification Protocol, and Impact Assessment.
Before (pp. 13-14)
V. Policy for Predetermined Change Control Plans 270 Software development is an iterative process, and FDA appreciates that manufacturers of device 271 software functions strive to continually improve and update their devices. Manufacturers shou
After (pp. 14-15)
V. Policy for Predetermined Change Control Plans Software development is an iterative process, and FDA appreciates that manufacturers of device software functions strive to continually improve and update their devices. Manufacturers should evalua
EXPANDED
A. Components of a PCCP
The new version significantly expands on the considerations for developing a PCCP, including specific details about intended use populations and environments, and more explicit requirements for the Description of Modifications and Impact Assessment.
Before (pp. 14-15)
A. Components of a PCCP 310 40 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/deciding-when-submit- 510k-software-change-existing-device. 41 Available at https://www.fda.gov/regulatory-information/search-
After (p. 15)
A. Components of a PCCP A PCCP should consist of a detailed Description of Modifications, a Modification Protocol, and an Impact Assessment (see Sections VI. – VIII. of this guidance) because these components are intended to provide FDA with infor
EXPANDED
B. Establishing a PCCP
The new version significantly expands on the types of marketing submissions appropriate for establishing a PCCP, particularly for AI-DSFs, and clarifies that submission types without an affirmative FDA decision are not suitable.
Before (pp. 15-16)
B. Establishing a PCCP 324 Premarket authorization for an ML-DSF with a PCCP must be established through the 510(k) 325 pathway, De Novo pathway, or PMA pathway, as appropriate, as a PCCP must be reviewed and 326 established as part of a marke
After (pp. 15-17)
B. Establishing a PCCP Premarket authorization for an AI-DSF with a PCCP must be established through the PMA pathway, 510(k) pathway, or De Novo pathway, as appropriate, as a PCCP must be reviewed and established as part of a marketing authorizati
EXPANDED
E. Modifying a Previously Authorized PCCP
The new version provides more specific recommendations for manufacturers on how to submit modifications to a PCCP, including providing a summary of changes and considering specific submission types like a special 510(k).
Before (pp. 19-20)
E. Modifying a PCCP for an Authorized Device 427 Because the modifications described in the PCCP include device changes that would otherwise 428 require a PMA supplement, De Novo submission, or new 510(k) premarket notification, at this 429 ti
After (pp. 22-23)
E. Modifying a Previously Authorized PCCP FDA believes that modifications to an authorized PCCP will generally constitute changes to the AI-DSF that would otherwise require a new marketing submission.77 In other words, FDA anticipates that modific
EXPANDED
VI. Description of Modifications
The new version expands on the description of modifications by adding a reference to appendices that provide examples and scenarios for implementing recommendations.
Before (p. 20)
VI. Description of Modifications 441 As introduced above, a description of each planned modification to an ML-DSF should be 442 included in the Description of Modifications section of a PCCP. The detailed description should 443 describe changes
After (p. 24)
VI. Description of Modifications A description of each planned modification to an AI-DSF should be included in the Description of Modifications section of a PCCP. The detailed description should describe specific changes to the device characterist
EXPANDED
C. Types of Modifications
The new version expands on the types of modifications related to device inputs, providing more specific examples and clarifying the scope of acceptable changes.
Before (pp. 21-22)
C. Types of Modifications 494 Modifications that are appropriate for a PCCP include those that are intended to maintain or 495 improve the safety or effectiveness of the device. Modifications proposed within the Description 496 of Modification
After (pp. 26-27)
C. Types of Modifications Modifications that are appropriate for inclusion in a PCCP include those that are intended to maintain or improve the safety or effectiveness of the device. Modifications should also be specific, and should be able to be
EXPANDED
VII. Modification Protocol
The new version expands on the content of the Modification Protocol by explicitly referencing Appendix A for detailed questions and considerations, and clarifies the QSR requirements for documenting changes.
Before (p. 22)
VII. Modification Protocol 531 The Modification Protocol includes the documentation describing the methods that will be 532 followed when developing, validating, and implementing modifications delineated in the 533 Description of Modifications
After (p. 27)
VII. Modification Protocol The Modification Protocol should include the documentation describing the methods that will be followed when developing, validating, and implementing modifications delineated in the Description of Modifications section o
EXPANDED
A. Goals of the Modification Protocol Section
The new version expands on the goals of the Modification Protocol by adding requirements for identifying the update process, communication/training plans, and ensuring anticipated risks and mitigation strategies are included in the Impact Assessment.
Before (pp. 22-24)
A. Goals of the Modification Protocol Section 544 Whereas the Description of Modifications outlines the planned modifications to an ML-DSF, the 545 Modification Protocol describes the methods that will be followed when developing, validating, 5
After (pp. 27-29)
A. Goals of the Modification Protocol Section Whereas the Description of Modifications outlines the planned modifications to an AI-DSF, the Modification Protocol should describe the methods that will be followed when developing, validating, and im
EXPANDED
B. Content of the Modification Protocol Section
The new version expands on data management practices by explicitly including 'tuning' data, clarifying data sequestration, and suggesting additional methods for bias mitigation.
Before (pp. 24-27)
B. Content of the Modification Protocol Section 602 As part of their Modification Protocol, manufacturers should outline the methods for each 603 component described below. Example elements of each of the four Modification Protocol 604 compone
After (pp. 29-33)
B. Content of the Modification Protocol Section To achieve these goals, FDA recommends that a Modification Protocol outline the methods for each component described below. Example elements of each of the four Modification Protocol components are p
EXPANDED
C. Traceability Between the Description of Modifications
The new version significantly expands on the example traceability table by providing a detailed, multi-column table with specific examples of methods and sections, whereas the old version only provided a textual description of the table's purpose.
Before (pp. 27-28)
C. Traceability Between the Description of Modifications 754 Section and the Modification Protocol Section 755 78 Section 515C(a)(3) and (b)(3) of the FD&C Act. 79 See 21 CFR 801.5, requiring that labeling include adequate directions for use
After (pp. 33-34)
C. Traceability Between the Description of Modifications Section and the Modification Protocol Section The PCCP should clearly delineate which parts of the Modification Protocol are applicable to each modification within the Description of Modific
EXPANDED
VIII. Impact Assessment
The new version expands on the requirements for an Impact Assessment by explicitly including 'unintended bias' as a risk to discuss and specifying 'verification and validation activities' within the Modification Protocol, and changing 'collective impact' to 'cumulative impact'.
Before (pp. 28-43)
VIII. Impact Assessment 770 An Impact Assessment, in the context of a PCCP, is the documentation of the assessment of the 771 benefits and risks of implementing a PCCP for an ML-DSF, as well as the mitigations of those 772 risks. The manufactu
After (pp. 34-49)
VIII. Impact Assessment An Impact Assessment in a PCCP is the documentation of the assessment of the benefits and risks of implementing a PCCP for an AI-DSF, as well as the mitigations of those risks. The manufacturer’s existing quality system sh
CLARIFICATION
I. Introduction
The new version clarifies the scope of the guidance by stating it's for 'AI-enabled devices' rather than specifically 'machine learning-enabled device software functions (ML-DSFs)', and removes the parenthetical definition of ML-DSFs.
Before (pp. 5-6)
I. Introduction 17 FDA has a longstanding commitment to develop and apply innovative approaches to the 18 regulation of medical device software and other digital health technologies to ensure their safety 19 and effectiveness.1 As technology c
After (pp. 5-6)
I. Introduction FDA has a longstanding commitment to develop and apply innovative approaches to the regulation of medical device software and other digital health technologies to ensure their safety and effectiveness.1 As technology continues to
CLARIFICATION
C. Predetermined Change Control Plan
The new version clarifies that PCCPs are specifically for AI-DSFs and updates the types of premarket submissions that would otherwise be required.
Before (p. 13)
C. Predetermined Change Control Plan 250 Predetermined Change Control Plan (PCCP): The documentation describing what 251 modifications will be made to the ML-DSF and how the modifications will be assessed. The 252 modifications described in th
After (pp. 13-14)
C. Predetermined Change Control Plan Predetermined Change Control Plan (PCCP): The documentation describing what modifications will be made to a device and how the modifications will be assessed. The modifications described in the PCCP include dev
CLARIFICATION
C. Identifying a PCCP in a Marketing Submission
The new version clarifies the required title and version number for the PCCP section and refines the language regarding references to the PCCP section and the assessment criteria.
Before (pp. 16-17)
C. Identifying a PCCP in a Marketing Submission 353 The PCCP should be included as a standalone section within the marketing submission. 354 Additionally, it should be prominently included and discussed in the cover letter and included in 355
After (pp. 17-19)
C. Identifying a PCCP in a Marketing Submission The PCCP should be included as a standalone section within the marketing submission, with a title and version number. Additionally, it should be prominently included and discussed in the cover letter
CLARIFICATION
D. Utilizing an Authorized PCCP to Implement Device
The new version clarifies the expectation for manufacturers to follow their authorized PCCP and applicable legal requirements, and explicitly states that FDA considers a modification consistent with the PCCP under specific conditions.
Before (pp. 17-19)
D. Utilizing an Authorized PCCP to Implement Device 380 Modifications 381 Once a PCCP has been reviewed and established through a marketing submission, the PCCP is 382 considered part of the marketing authorization. In general, a PCCP should b
After (pp. 19-22)
D. Utilizing an Authorized PCCP to Implement Device Modifications FDA expects manufacturers to follow their authorized PCCP, and manufacturers are required to follow applicable legal requirements regarding the device and its authorized PCCP. In ge
CLARIFICATION
B. Content of the Description of Modifications Section
The new version clarifies the reference to labeling changes by specifying 'labeling sections' and providing a more precise section number for the Modification Protocol.
Before (pp. 20-21)
B. Content of the Description of Modifications Section 460 The Description of Modifications should enumerate the list of individual proposed device 461 modifications discussed in the PCCP, as well as the specific rationale for the change to each
After (pp. 25-26)
B. Content of the Description of Modifications Section To achieve these goals, FDA recommends that the Description of Modifications enumerate the list of individual proposed device modifications discussed in the PCCP, as well as the specific ratio
EDITORIAL
IV. Definitions
The line numbers '217' and '218' were removed from the new version.
Before (p. 12)
IV. Definitions 217 This section defines certain terms as they are used for the purposes of this guidance. 218
After (p. 12)
IV. Definitions This section defines certain terms as they are used for purposes of this guidance
Total Changes
49
High Impact
47
Change Breakdown
NEW (7)
STRICTER (1)
EXPANDED (2)
RESTRUCTURED (3)
REMOVED (36)
Filter
What do these change types mean?
NEW — Content only in newer document
REMOVED — Content only in older document
STRICTER — Requirements more stringent
MORE_LENIENT — Requirements less stringent
EXPANDED — Same topic, more detail added
CLARIFICATION — Same meaning, clearer language
RESTRUCTURED — Content moved/reorganized
EDITORIAL — Cosmetic changes only
EXPANDED
HIGH IMPACT
III. Scope
The new version significantly expands the definition of 'software devices' by introducing the concept of 'device software functions' and providing a detailed explanation of what constitutes a 'function' within a product, including examples.
Before (pp. 6-7)
Scope For the purposes of this document, we refer to devices that contain one or more software components, parts, or accessories, or are composed solely of software as “software devices,” including: • firmware and other means for software-based
After (pp. 7-9)
III. Scope For the purposes of this document, FDA refers to a software function that meets the definition of a device as a device software function. For any given product, the term “function” is a distinct purpose of the product, which could be
EXPANDED
HIGH IMPACT
3. A cloud-based device algorithm for analyzing previously captured medical images
The new version significantly expands on the concept of architecture design documentation, moving from a brief mention of 'detailed depiction' to providing extensive examples, figures, and guidance on the content and presentation of system and software architecture diagrams.
Before (p. 13)
Architecture Design Chart No documentation is necessary in the submission. Detailed depiction of functional units and software modules. May include state diagrams as well as flow charts
After (pp. 42-45)
3. A cloud-based device algorithm for analyzing previously captured medical images The diagrams are largely static diagrams with high-level identification of interfaces between system and software components. The use of any specific design or format
NEW
HIGH IMPACT
I. Introduction
New content: I. Introduction
No previous content
After (pp. 4-5)
I. Introduction This guidance document is intended to provide information regarding the recommended documentation for premarket submissions for FDA’s evaluation of the safety and effectiveness of device software functions, which are software fun
NEW
HIGH IMPACT
IV. Definitions
New content: IV. Definitions
No previous content
After (pp. 9-11)
IV. Definitions The following terms are used for the purposes of this guidance: Device Software Function - Software function that meets the device definition in section 201(h) of the FD&C Act. As discussed above, the term “function” is a distin
NEW
HIGH IMPACT
V. Documentation Level
New content: V. Documentation Level
NEW
HIGH IMPACT
VI. Recommended Documentation
New content: VI. Recommended Documentation
No previous content
After (pp. 12-31)
VI. Recommended Documentation This section reflects FDA’s recommendations for information to be included in premarket submissions for Basic and Enhanced Documentation Levels. This recommended information 31 For information on risk assessment refer
NEW
HIGH IMPACT
VII. Additional Information Regulatory Considerations for
New content: VII. Additional Information Regulatory Considerations for
No previous content
After (pp. 31-32)
VII. Additional Information Regulatory Considerations for Software Functions Section 3060(a) of the Cures Act amended section 520 of the FD&C Act on December 13, 2016, removing certain software functions from the definition of device in section
NEW
HIGH IMPACT
20. An infusion pump intended for use in a health care facility to pump fluids and
New content: 20. An infusion pump intended for use in a health care facility to pump fluids and
No previous content
After (pp. 38-42)
20. An infusion pump intended for use in a health care facility to pump fluids and medications into a patient. Description: The device is intended for use on adults, pediatrics, and neonates for the intermittent or continuous delivery of fluids
NEW
HIGH IMPACT
1. A hand-held diagnostic device
New content: 1. A hand-held diagnostic device
No previous content
After (p. 42)
1. A hand-held diagnostic device
REMOVED
HIGH IMPACT
Introduction
Removed: Introduction
Before (p. 5)
Introduction This guidance document is intended to provide information to industry regarding the documentation that we recommend you include in premarket submissions for software devices, including stand alone software applications and hardware-ba
Content removed
REMOVED
HIGH IMPACT
The Least Burdensome Approach
Removed: The Least Burdensome Approach
Before (pp. 5-6)
The Least Burdensome Approach The issues identified in this guidance document represent those that we believe should be addressed before your device can be marketed. In developing the guidance, we carefully considered the relevant statutory criter
Content removed
REMOVED
HIGH IMPACT
Software-Related Consensus Standards
Removed: Software-Related Consensus Standards
Before (p. 7)
Software-Related Consensus Standards The emergence of consensus standards related to software has helped to improve the consistency and quality of software development and documentation, particularly with respect to critical activities such as ris
Content removed
REMOVED
HIGH IMPACT
Terminology Verification and Validation
Removed: Terminology Verification and Validation
Before (pp. 7-8)
Terminology Verification and Validation This document uses the terms "verification" and "validation" (also referred to as “V&V”) as they are defined in the QS regulation.iv Verification “means confirmation by examination and provision of objectiv
Content removed
REMOVED
HIGH IMPACT
Minor and Serious Injuries
Removed: Minor and Serious Injuries
Before (p. 8)
Minor and Serious Injuries For the purposes of this document, we use the term minor injury to mean any injury that does not meet the definition of a serious injury as defined in 21 CFR 803.3(bb)(1). This regulation defines serious injury as an inj
Content removed
REMOVED
HIGH IMPACT
Major
Removed: Major
Before (p. 9)
Major We believe the level of concern is Major if a failure or latent flaw could directly result in death or serious injury to the patient or operator. The level of concern is also Major if a failure or latent flaw could indirectly result in death
Content removed
REMOVED
HIGH IMPACT
Minor
Removed: Minor
Before (pp. 9-10)
Minor We believe the level of concern is Minor if failures or latent design flaws are unlikely to cause any injury to the patient or operator. 5 Contains Nonbinding Recommendations Determining
Content removed
REMOVED
HIGH IMPACT
Level of Concern
Removed: Level of Concern
Before (p. 14)
Level of Concern We recommend that you indicate the Level of Concern for your Software Device, determined before the effects of any mitigations. We recommend that you clearly state which one of the three levels of concern is appropriate for your
Content removed
REMOVED
HIGH IMPACT
Software Description
Removed: Software Description
Before (pp. 14-15)
Software Description We recommend that you provide a comprehensive overview of the device features that are controlled by software, and describe the intended operational environment. Generally, we 10
Content removed
REMOVED
HIGH IMPACT
Device Hazard Analysis
Removed: Device Hazard Analysis
Before (p. 15)
Device Hazard Analysis We recommend that you submit a Device Hazard Analysis for all Software Devices. The Device Hazard Analysis should take into account all device hazards associated with the device’s intended use, including both hardware and s
Content removed
REMOVED
HIGH IMPACT
Software Requirements
Removed: Software Requirements
Before (p. 13)
Software Requirements Specification (SRS)
Content removed
REMOVED
HIGH IMPACT
Summary of
Removed: Summary of
Before (p. 13)
Summary of software life cycle development plan, including a summary of the configuration management and
Content removed
REMOVED
HIGH IMPACT
Software Design
Removed: Software Design
Before (p. 13)
Software Design Specification (SDS)
Content removed
REMOVED
HIGH IMPACT
No
Removed: No
Before (p. 13)
No documentation is necessary in the submission. Software design specification document
Content removed
REMOVED
HIGH IMPACT
Traceability Analysis
Removed: Traceability Analysis
Before (pp. 17-18)
Traceability Analysis A Traceability Analysis links together your product design requirements, design specifications , and testing requirements. It also provides a means of tying together identified hazards with the implementation and testing of
Content removed
REMOVED
HIGH IMPACT
Software Development Environment Description No
Removed: Software Development Environment Description No
Before (p. 13)
Software Development Environment Description No documentation is necessary in the submission
Content removed
REMOVED
HIGH IMPACT
Annotated list of
Removed: Annotated list of
Before (pp. 13-14)
Annotated list of control documents generated during development process. Include the 9 Contains Nonbinding Recommendations SOFTWARE DOCUMENTATION MINOR CONCERN MODERATE CONCERN MAJOR
Content removed
REMOVED
HIGH IMPACT
Documentation
Removed: Documentation
Before (p. 14)
Documentation plan, pass / fail criteria, and results. the unit, integration, and system level. System level test protocol, including pass/fail criteria, and tests results. the unit, integration, and system level. Unit, integration and
Content removed
REMOVED
HIGH IMPACT
Revision Level History
Removed: Revision Level History
Before (pp. 19-21)
Revision Level History Your submission should include the history of software revisions generated during the course of product development. This typically takes the form of a line-item tabulation of the major changes to the software during the dev
Content removed
REMOVED
HIGH IMPACT
Unresolved Anomalies No
Removed: Unresolved Anomalies No
Before (p. 14)
Unresolved Anomalies No List of remaining software anomalies, (Bugs or Defects) documentation is necessary in the submission. annotated with an explanation of the impact on safety or effectiveness, including operator usage and human factors
Content removed
REMOVED
HIGH IMPACT
Software Requirements Specification
Removed: Software Requirements Specification
Before (pp. 15-16)
Software Requirements Specification The Software Requirements Specification (SRS) documents the requirements for the software. This typically includes functional, performance, interface, design, developmental, and other requirements for the softw
Content removed
REMOVED
HIGH IMPACT
Hardware Requirements
Removed: Hardware Requirements
Before (p. 16)
Hardware Requirements Hardware requirements generally include: • microprocessors • memory devices • sensors • energy sources • safety features • communications
Content removed
REMOVED
HIGH IMPACT
Programming Language Requirements
Removed: Programming Language Requirements
Before (p. 16)
Programming Language Requirements Programming language requirements include program size requirements or restrictions, and information on management of memory leaks
Content removed
REMOVED
HIGH IMPACT
Interface Requirements
Removed: Interface Requirements
Before (p. 16)
Interface Requirements Interface requirements generally include both communication between system components and communication with the user such as: • printers • monitors • keyboard • mouse
Content removed
REMOVED
HIGH IMPACT
Software Performance and Functional Requirements
Removed: Software Performance and Functional Requirements
Before (pp. 16-17)
Software Performance and Functional Requirements Software performance and functional requirements include algorithms or control characteristics for therapy, diagnosis, monitoring, alarms, analysis, and interpretation with full text references or s
Content removed
REMOVED
HIGH IMPACT
Architecture Design Chart
Removed: Architecture Design Chart
Before (p. 17)
Architecture Design Chart This document is typically a flowchart or similar depiction of the relationships among the major functional units in the Software Device, including relationships to hardware and to data flows such as networking. It is usu
Content removed
REMOVED
HIGH IMPACT
Software Design Specification
Removed: Software Design Specification
Before (p. 17)
Software Design Specification The Software Design Specification (SDS) describes the implementation of the requirements for the Software Device. In terms of the relationship between the SRS and the SDS, the SRS describes what the Software Device wi
Content removed
REMOVED
HIGH IMPACT
Software Development Environment Description
Removed: Software Development Environment Description
Before (p. 18)
Software Development Environment Description For Moderate and Major Level of Concern Software Devices, the submission should include a summary of the software development life cycle plan. This summary should describe the sponsor’s software develop
Content removed
REMOVED
HIGH IMPACT
Verification and Validation Documentation
Removed: Verification and Validation Documentation
Before (p. 18)
Verification and Validation Documentation The terms “verification” and “validation” described earlier in this document refer to two phases of Software Device testing. This section recommends the type of testing documentation you should include in
Content removed
REMOVED
HIGH IMPACT
Minor Level of Concern Devices
Removed: Minor Level of Concern Devices
Before (p. 18)
Minor Level of Concern Devices For Minor Level of Concern devices, we recommend that you submit documentation of system or device level testing, and, where appropriate, integration testing. The documentation submitted should include system or dev
Content removed
REMOVED
HIGH IMPACT
Moderate Level of Concern Devices
Removed: Moderate Level of Concern Devices
Before (pp. 18-19)
Moderate Level of Concern Devices For Moderate Level of Concern devices, we recommend that you submit a summary list of validation and verification activities and the results of these activities. We also recommend that you submit your pass/fail cr
Content removed
REMOVED
HIGH IMPACT
Risk Assessment and Management Background
Removed: Risk Assessment and Management Background
Before (p. 21)
Risk Assessment and Management Background Inadequate or inappropriate software development life cycle and risk management activities, inappropriate use of a Software Device, or operational errors can result in a variety of potential failures or d
Content removed
REMOVED
HIGH IMPACT
Risk Assessment and Level of Concern
Removed: Risk Assessment and Level of Concern
Before (p. 21)
Risk Assessment and Level of Concern As mentioned earlier, your assessment of the risks associated with your Software Device should assist you in determining an appropriate Level of Concern. We also recommend that you consider the Level of Concer
Content removed
REMOVED
HIGH IMPACT
Risk Management
Removed: Risk Management
Before (p. 21)
Risk Management The risk associated with Software Devices varies over a continuum from negligible to very severe. In general, FDA considers risk as the product of the severity of injury and the probability of its occurrence. However, software fail
Content removed
REMOVED
HIGH IMPACT
Software Change Management
Removed: Software Change Management
Before (pp. 21-22)
Software Change Management Design, development, testing, and version control of revisions to the software are as important as development and testing of the software that was reviewed in the premarket submission. We believe the majority of softwar
Content removed
REMOVED
HIGH IMPACT
Virus Protection Software
Removed: Virus Protection Software
Before (pp. 22-23)
Virus Protection Software Software applications designed to protect information systems, including Software Devices, from harmful or malicious code (“viruses,” “worms,” etc.) are becoming more commonplace as devices become increasingly interconnec
Content removed
RESTRUCTURED
HIGH IMPACT
2. An implantable therapeutic device with patient- and provider-facing applications
The old version defines 'Moderate' level of concern based on potential for minor injury, while the new version provides an example of a device type. These passages do not cover the same regulatory topic.
Before (p. 9)
Moderate We believe the level of concern is Moderate if a failure or latent design flaw could directly result in minor injury to the patient or operator. The level of concern is also Moderate if a failure or latent flaw could indirectly result in
After (p. 42)
2. An implantable therapeutic device with patient- and provider-facing applications
STRICTER
HIGH IMPACT
V. Documentation Level
The concept of 'Level of Concern' has been replaced with 'Documentation Level' (Basic or Enhanced), and the criteria for Enhanced Documentation are more explicitly defined based on probable risk of death or serious injury, assessed prior to risk control measures, making the requirements more stringent.
Before (pp. 8-9)
Level of Concern Introduction The documentation that we recommend you include in a premarket submission generally depends on the device’s Level of Concern. For the purposes of this guidance document, Level of Concern refers to an estimate of the
After (pp. 11-12)
V. Documentation Level The recommended documentation for a premarket submission depends on the device’s risk to a patient, a user of a device, or others in the environment of use. FDA intends to take a risk-based approach to help determine the d
RESTRUCTURED
2. A non-contact infrared thermometer intended for intermittent measurement of body
The old version presented a series of questions to determine the level of concern for software devices, while the new version provides specific examples of devices with their descriptions, rationales, and outcomes.
Before (pp. 10-13)
1. Does the Software Device qualify as Blood Establishment Computer Software? (Blood Establishment Computer Software is defined as software products intended for use in the manufacture of blood and blood components or for the maintenance of data t
After (pp. 32-38)
2. A non-contact infrared thermometer intended for intermittent measurement of body temperature from the forehead. Description: The device is intended to measure body temperature from the forehead using an infrared sensor. The device is a hand-
RESTRUCTURED
II. Background
The 'References' section from the old version has been replaced with a 'Background' section in the new version, which now includes a narrative about the purpose of the guidance and its relation to other documents and legislative changes, rather than just a list of references.
Before (pp. 23-24)
References i This document combines the recommendations in “Guidance for FDA Reviewers and Industry: Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices” issued on May 29, 1998, and “Reviewer Guidance for
After (pp. 5-7)
II. Background The purpose of this guidance is to describe FDA’s thinking on the recommended documentation sponsors should include in premarket submissions for FDA’s evaluation of the safety and effectiveness of device software functions. This t
How It Works
The pipeline extracts text from FDA guidance PDFs, chunks them by section hierarchy, aligns chunks across document versions using BM25 retrieval, then classifies each change with an LLM. The result is a structured diff that tells you not just what changed, but how: stricter requirements, new content, clarifications, or removals.
Key Technical Decisions
| Component | Choice | Why |
|---|---|---|
| Alignment | BM25 (sparse) | 0.915 MRR on 116 labeled queries. Regulatory terminology is stable across years—keyword matching outperformed embeddings. |
| Chunking | Parent-child | Best MRR (0.862) while preserving document hierarchy. 53 chunks vs 545 for fixed-size baseline. |
| Classification | Gemini 2.5 Flash | Consistent taxonomy output. 100% valid JSON on classification calls. |
| Thresholds | MATCH=15.0, NO_MATCH=5.0 | Tuned on ground truth—scores above 15 reliably indicate same content. |
Validation Results
Validated against law firm analyses (King & Spalding cybersecurity and software premarket reviews).
| Document Pair | Gap | Detection | Type Accuracy |
|---|---|---|---|
| Cybersecurity 2014→2023 | 9 years | 6/6 (100%) | 6/6 (100%) |
| PCCP AI 2023→2024 | 1.5 years | 9/10 (90%) | 9/9 (100%) |
| Software Premarket 2005→2023 | 18 years | 3/4 (75%) | 2/3 (67%) |
Modern FDA documents (post-2010) with numbered section headers achieve 90-100% detection. Older documents with inconsistent formatting are harder—75% detection on the 2005 software guidance.
Change Taxonomy
The system classifies changes into eight types, separating what the LLM determines (matched content) from what alignment determines (unmatched content):
LLM-Classified (Matched Pairs)
- STRICTER — Requirements more stringent
- MORE_LENIENT — Requirements relaxed
- EXPANDED — Same topic, more detail
- CLARIFICATION — Same meaning, clearer
- RESTRUCTURED — Content moved
- EDITORIAL — Cosmetic only
Alignment-Determined (No Match)
- NEW — Content only in newer document
- REMOVED — Content only in older document
Architecture
PDF → FDAChunker (parent_child) → Chunks
↓
Old chunks + New chunks → BM25Index.align_chunks() → ChunkMatches
↓
ChunkMatches → LLM classification → ClassifiedChanges → JSON Interested in learning more?
Check out my other projects or get in touch.