Skip to content

Development Evolution

Jump to bottom
George M. Dias edited this page Mar 29, 2023 · 10 revisions
updates.ts

Current Algorithm

The approach is to scan all lines, and collapsing them with the associate header. Algorithm supports the following control headers:

control, title, desc, impact, tag, and ref To be a valid Profile Control the control header must be the parent header, there is, all other headers are contained inside the control header:

control 'control_name' do
  title ...
  desc`...
  .
  .
  .
  impact [value]
  tag ...
  .
  .
  .
end

order is not important

To collapse all content to the associated headers the algorithm utilizes a pair of stacks (i.e., stack, rangeStack) to keep track of string delimiters and their associated line numbers, respectively.

The algorithm handles the following delimiters:

  • Single quotes (')
  • Double quotes (")
  • Back ticks (`)
  • Mixed quotes ("`'")
  • Percent strings (%; keys: q, Q, r, i, I, w, W, x; delimiters: (), {}, [], <>, most non-alphanumeric characters); (e.g., "%q()")
  • Percent literals (%; delimiters: (), {}, [], <>, most non- alphanumeric characters); (e.g., "%()")
  • Multi-line comments (e.g., =begin\nSome comment\n=end)
  • Variable delimiters (i.e., parenthesis: (); array: []; hash: {})

An example how the algorithm work is a s follow, given the text below it would create the indicated stacks then collapsing each header leaving any text not belonging to a header as the describe block

stack[]        (string)   -> holds the delimiters (i.e., ', {, (, etc)
rangeStack[][] (int,int)  -> holds the line numbers where the delimiters were found (start - end)
ranges[][]     (int,int)  -> holds the accumulative location pairs

Example control:

image

Stack Action Value Line #
rangeStack Push [ "'" ] 1
stack Pop [] 1
rangeStack Push [ "'" ] 2
stack Pop [] 2
rangeStack Push [ "'" ] 3
stack Pop [] 3
rangeStack Push [ '"' ] 3
rangeStack Push [ '"', '{' ] 8
stack Pop [ '"' ] 8
stack Pop [] 10
rangeStack Push [ "'" ] 12
stack Pop [] 12
rangeStack Push [ '"' ] 12
rangeStack Push [ '"', '{' ] 14
stack Pop [ '"' ] 14
stack Pop [] 16
rangeStack Push [ "'" ] 18
stack Pop [] 18
rangeStack Push [ "'" ] 19
stack Pop [] 19
rangeStack Push [ "'" ] 19
stack Pop [] 19
rangeStack Push [ "'" ] 20
stack Pop [] 20
rangeStack Push [ '[' ] 20
stack Pop [] 20
rangeStack Push [ "'" ] 21
stack Pop [] 21
rangeStack Push [ "'" ] 21
stack Pop [] 21
rangeStack Push [ "'" ] 22
stack Pop [] 22
rangeStack Push [ "'" ] 22
stack Pop [] 22
rangeStack Push [ "'" ] 23
stack Pop [] 23
rangeStack Push [ "'" ] 23
stack Pop [] 23
rangeStack Push [ "'" ] 24
stack Pop [] 24
rangeStack Push [ "'" ] 24
stack Pop [] 24
rangeStack Push [ "'" ] 25
stack Pop [] 25
rangeStack Push [ '[' ] 25
stack Pop [] 25
rangeStack Push [ "'" ] 26
stack Pop [] 26
rangeStack Push [ '[' ] 26
stack Pop [] 26
rangeStack Push [ '(' ] 28
stack Pop [] 28
rangeStack Push [ '{' ] 29
stack Pop [] 29
rangeStack Push [ '(' ] 30
stack Pop [] 30
rangeStack Push [ '{' ] 30
stack Pop [] 30

The ranges locations array (accumulative location pairs) generated by the getRangesForLines function consists of:

    [
      [ 0, 0 ],   [ 1, 1 ],   [ 2, 2 ],
      [ 2, 9 ],   [ 11, 11 ], [ 11, 15 ],
      [ 17, 17 ], [ 18, 18 ], [ 18, 18 ],
      [ 19, 19 ], [ 19, 19 ], [ 20, 20 ],
      [ 20, 20 ], [ 21, 21 ], [ 21, 21 ],
      [ 22, 22 ], [ 22, 22 ], [ 23, 23 ],
      [ 23, 23 ], [ 24, 24 ], [ 24, 24 ],
      [ 25, 25 ], [ 25, 25 ], [ 27, 27 ],
      [ 28, 28 ], [ 29, 29 ], [ 29, 29 ]
    ]

The transformation to only multi-lines array generated by getMultiLineRanges function consists of:

    [ [ 2, 9 ], [ 11, 15 ] ]

The assembled code generated by the joinMultiLineStringsFromRanges function consists of:

    [
      "control 'V-93149' do",
      "  title 'Windows Server 2019 title for legal banner dialog box must be configured with the appropriate text.'",
      `  desc  'check', "If the following registry value does not exist or is not configured as specified, this is a finding:\n` +
        '\n' +
        '  Value Type: REG_SZ\n' +
        '    Value: See message title options below\n' +
        '\n' +
        `    \\"#{input('LegalNoticeCaption').join('", "')}\\", or an organization-defined equivalent.\n` +
        '\n' +
        '    If an organization-defined title is used, it can in no case contravene or modify the language of the banner text required in WN19-SO-000150."',
      '',
      `  desc  'fix', "Configure the policy value for Computer Configuration >> Windows Settings >> \n` +
        '  Security Settings >> Local Policies >> Security Options >> \\"Interactive Logon: Message title for users attempting \n' +
        `  to log on\\" to \\"#{input('LegalNoticeCaption').join('", "')}\\", or an organization-defined equivalent.\n` +
        '\n' +
        '    If an organization-defined title is used, it can in no case contravene or modify the language of the message text required in WN19-SO-000150."',
      '  impact 0.3',
      "  tag 'severity': nil",
      "  tag 'gtitle': 'SRG-OS-000023-GPOS-00006'",
      "  tag 'satisfies': ['SRG-OS-000023-GPOS-00006', 'SRG-OS-000228-GPOS-00088']",
      "  tag 'gid': 'V-93149'",
      "  tag 'rid': 'SV-103237r1_rule'",
      "  tag 'stig_id': 'WN19-SO-000140'",
      "  tag 'fix_id': 'F-99395r1_fix'",
      "  tag 'cci': ['CCI-000048', 'CCI-001384', 'CCI-001385', 'CCI-001386', 'CCI-001387', 'CCI-001388']",
      "  tag 'nist': ['AC-8 a', 'AC-8 c 1', 'AC-8 c 2', 'AC-8 c 2', 'AC-8 c 2', 'AC-8 c 3', 'Rev_4']",
      '',
      "  describe registry_key('HKEY_LOCAL_MACHINE\\\\SOFTWARE\\\\Microsoft\\\\Windows\\\\CurrentVersion\\\\Policies\\\\System') do",
      "    it { should have_property 'LegalNoticeCaption' }",
      "    its('LegalNoticeCaption') { should be_in input('LegalNoticeCaption') }",
      '  end',
      'end',
      ''
    ]

Notice that the lines are concatenated (see the +) such that each header has a long line with all it's associated text.

Option 1 (not implemented) 
export function getExistingDescribeFromControl1(control: Control): string {
   // Algorithm:
   //   Locate the start and end of the control string
   //   Update the end of the control that contains information (if empty lines are at the end of the control)
   //   loop: until the start index is changed (loop is done from the bottom up)
   //     Clean testing array entry line (removes any non-print characters)
   //     if: line starts with meta-information 'tag' or 'ref'
   //       set start index to found location
   //       break out of the loop
   //     end
   //   end
   //   Remove any empty lines after the start index (in any)
   //   Extract the describe block from the audit control given the start and end indices
   // Assumptions: 
   //  1 - The meta-information 'tag' or 'ref' precedes the describe block
   // Pros: Solves the potential issue with option 1, as the lookup for the meta-information
   //       'tag' or 'ref' is expected to the at the beginning of the line.
   if (control.code) {
     let existingDescribeBlock = ''
     let indexStart = control.code.toLowerCase().indexOf('control')
     let indexEnd = control.code.toLowerCase().trimEnd().lastIndexOf('end')
     const auditControl = control.code.substring(indexStart, indexEnd).split('\n')
 
     indexStart = 0
     indexEnd = auditControl.length - 1
     indexEnd = getIndexOfFirstLine(auditControl, indexEnd, '-')
     let index = indexEnd
 
     while (indexStart === 0) {
      // Look back 2 lines - Original looked behind 1 line
       const line = auditControl[index-1].toLowerCase().trim()
       if (line.indexOf('ref ') === 0 || line.indexOf('tag ') === 0 || line.indexOf('desc ') === 0) {
          console.log('LINE IS: ', line)
          indexStart = index + 1
       }
       index--
     }
 
     indexStart = getIndexOfFirstLine(auditControl, indexStart, '+')
     existingDescribeBlock = auditControl.slice(indexStart, indexEnd + 1).join('\n').toString()
 
     return existingDescribeBlock
   } else {
     return ''
   }
 }

Option 1 supporting function

/*
  Return first index found from given array that is not an empty entry (cell)
*/
function getIndexOfFirstLine(auditArray: string[], index: number, action: string): number {
  let indexVal = index;
  while (auditArray[indexVal] === '') {
    switch (action) {
      case '-':
        indexVal--
        break;
      case '+':
        indexVal++
        break;
    }
  }

  return indexVal
}
Option 2 (not implemented) 
function getExistingDescribeFromControl(control: Control): string {
  // Algorithm: 
  //   Locate the index of the last occurrence of the meta-information 'tag'
  //   if: we have a tag do
  //     Place each line of the control code into an array
  //     loop: over the array starting at the end of the line the last meta-information 'tag' was found
  //       remove any empty before describe block content is found
  //       add found content to describe block variable, append EOL
  //     end
  //   end
  // Assumptions: 
  //  1 - The meta-information 'tag' precedes the describe block
  // Potential Problems:
  //  1 - The word 'tag' could be part of the describe block
  if (control.code) {
    let existingDescribeBlock = ''
    const lastTag = control.code.lastIndexOf('tag')
    if (lastTag > 0) {
      const tagEOL = control.code.indexOf('\n',lastTag)
      const lastEnd = control.code.lastIndexOf('end')    
      let processLine = false
      control.code.substring(tagEOL,lastEnd).split('\n').forEach((line) => {
        // Ignore any blank lines at the beginning of the describe block
        if (line !== '' || processLine) {
          existingDescribeBlock += line + '\n'
          processLine = true
        }
      })      
    }
    return existingDescribeBlock.trimEnd();
  } else {
    return ''
  }
}
The original code 
function getExistingDescribeFromControl(control: Control): string {
  if (control.code) {
  let existingDescribeBlock = ''
  let currentQuoteEscape = ''
  const percentBlockRegexp = /%[qQrRiIwWxs]?(?<lDelimiter>[([{<])/;
  let inPercentBlock = false;
  let inQuoteBlock = false
  const inMetadataValueOverride = false
  let indentedMetadataOverride = false
  let inDescribeBlock = false;
  let mostSpacesSeen = 0;
  let lDelimiter = '(';
  let rDelimiter = ')';

  control.code.split('\n').forEach((line) => {
    const wordArray = line.trim().split(' ')
    const spaces = line.substring(0, line.indexOf(wordArray[0])).length

    if (spaces - mostSpacesSeen  > 10) {
      indentedMetadataOverride = true
    } else {
      mostSpacesSeen = spaces;
      indentedMetadataOverride = false
    }

    if ((!inPercentBlock && !inQuoteBlock && !inMetadataValueOverride && !indentedMetadataOverride) || inDescribeBlock) {
      if (inDescribeBlock && wordArray.length === 1 && wordArray.includes('')) {
        existingDescribeBlock += '\n'
      }
      // Get the number of spaces at the beginning of the current line
      else if (spaces >= 2) {
        const firstWord = wordArray[0]
        if (knownInSpecKeywords.indexOf(firstWord.toLowerCase()) === -1 || (knownInSpecKeywords.indexOf(firstWord.toLowerCase()) !== -1 && spaces > 2) || inDescribeBlock) {
          inDescribeBlock = true;
          existingDescribeBlock += line + '\n'
        }
      }
    }

    wordArray.forEach((word, index) => {
      const percentBlockMatch = percentBlockRegexp.exec(word); 
      if(percentBlockMatch && inPercentBlock === false) {
        inPercentBlock = true;
        // eslint-disable-next-line  @typescript-eslint/no-non-null-assertion
        lDelimiter = percentBlockMatch.groups!.lDelimiter || '(';
        switch(lDelimiter) { 
          case '{': { 
            rDelimiter = '}';
            break; 
          } 
          case '[': { 
            rDelimiter = ']';
            break; 
          } 
          case '<': { 
            rDelimiter = '>';
            break; 
          } 
          default: { 
            break; 
          } 
        }
                    
      }
      const charArray = word.split('')
      charArray.forEach((char, index) => {
        if (inPercentBlock) {
          if (char === rDelimiter && charArray[index - 1] !== '\\' && !inQuoteBlock) {
            inPercentBlock = false;
          }
        }
        if (char === '"' && charArray[index - 1] !== '\\') {
          if (!currentQuoteEscape || !inQuoteBlock) {
            currentQuoteEscape = '"'
          }
          if (currentQuoteEscape === '"') {
            inQuoteBlock = !inQuoteBlock
          }
        } else if (char === "'" && charArray[index - 1] !== '\\') {
          if (!currentQuoteEscape || !inQuoteBlock) {
            currentQuoteEscape = "'"
          }
          if (currentQuoteEscape === "'") {
            inQuoteBlock = !inQuoteBlock
          }
        }
      })
    })
  })
  // Take off the extra newline at the end
  return existingDescribeBlock.slice(0, -1)
  } else {
    return ''
  }
}
diffMarkdown.ts Removed unused function: 
function getUpdatedCheckForId(id: string, profile: Profile) {
  const foundControl = profile.controls.find((control) => control.id === id);
  return _.get(foundControl?.descs, 'check') || 'Missing check';
}
global.ts Removed unused function: 
const wrapAndEscapeQuotes = (s: string, lineLength?: number) =>
  escapeDoubleQuotes(wrap(s, lineLength)); // Escape backslashes and quotes, and wrap long lines

Typescript 🧾 objects 🎛️ for InSpec 🔎 profiles 📄

image

InSpec Object

Support utility for security automation

Table of Contents

Common Instructions

Developers Corner

Clone this wiki locally