Baike.dev
Log in
Back to tool/Back to issues
#27·mini-vue

【看的见的思考】compiler-core

Author: cuixiaoruiCreated Aug 24, 2021Updated Aug 26, 2021

看的见的思考

先从compile 看起， compile 里面的 baseCompile 是整个 core 的入口函数

先看看单元测试吧

javascript
  test('function mode', () => {
    const { code, map } = compile(source, {
      sourceMap: true,
      filename: `foo.vue`
    })

    expect(code).toMatchSnapshot()
    expect(map!.sources).toEqual([`foo.vue`])
    expect(map!.sourcesContent).toEqual([source])

    const consumer = new SourceMapConsumer(map as RawSourceMap)

    expect(
      consumer.originalPositionFor(getPositionInCode(code, `id`))
    ).toMatchObject(getPositionInCode(source, `id`))

这个测试相当的大，先一点点的看

先看input source 是

javascript
  const source = `
<div id="foo" :class="bar.baz">
  {{ world.burn() }}
  <div v-if="ok">yes</div>
  <template v-else>no</template>
  <div v-for="(value, index) in list"><span>{{ value + index }}</span></div>
</div>
`.trim()

这个就是 template 了。 然后第二个参数是对应的 CompilerOptions 先不用管

在去看看输出 output

下面是 code

javascript
 const _Vue = Vue
    
    return function render(_ctx, _cache) {
      with (_ctx) {
        const { toDisplayString: _toDisplayString, openBlock: _openBlock, createElementBlock: _createElementBlock, createCommentVNode: _createCommentVNode, createTextVNode: _createTextVNode, Fragment: _Fragment, renderList: _renderList, createElementVNode: _createElementVNode, normalizeClass: _normalizeClass } = _Vue
    
        return (_openBlock(), _createElementBlock("div", {
          id: "foo",
          class: _normalizeClass(bar.baz)
        }, [
          _createTextVNode(_toDisplayString(world.burn()) + " ", 1 /* TEXT */),
          ok
            ? (_openBlock(), _createElementBlock("div", { key: 0 }, "yes"))
            : (_openBlock(), _createElementBlock(_Fragment, { key: 1 }, [
                _createTextVNode("no")
              ], 2112 /* STABLE_FRAGMENT, DEV_ROOT_FRAGMENT */)),
          (_openBlock(true), _createElementBlock(_Fragment, null, _renderList(list, (value, index) => {
            return (_openBlock(), _createElementBlock("div", null, [
              _createElementVNode("span", null, _toDisplayString(value + index), 1 /* TEXT */)
            ]))
          }), 256 /* UNKEYED_FRAGMENT */))
        ], 2 /* CLASS */))
      }
    }

就是生成的 render 函数

那map 是什么？

javascript
  {
      version: 3,
      sources: [ 'foo.vue' ],
      names: [],
      mappings: ';;;;;;0BAAA,oBAKM;MALD,EAAE,EAAC,KAAK;MAAE,KAAK,EAApB,gBAAsB,OAAO;;MAA7B,kCACK,YAAY,IAAG,GAClB;MAAW,EAAE;yBAAb,oBAAwB,SAF1B,KAAA,KAEiB,KAAG;yBAClB,oBAA8B,aAHhC,KAAA;YAAA,iBAGmB,IAAE;;yBACnB,oBAA0E,iBAJ5E,YAIgC,IAAI,EAJpC,CAIe,KAAK,EAAE,KAAK;8BAAzB,oBAA0E;UAAtC,oBAAgC,+BAAvB,aAAa',
      sourcesContent: [
        '<div id="foo" :class="bar.baz">\n' +
          '  {{ world.burn() }}\n' +
          '  <div v-if="ok">yes</div>\n' +
          '  <template v-else>no</template>\n' +
          '  <div v-for="(value, index) in list"><span>{{ value + index }}</span></div>\n' +
          '</div>'
      ]
    }

应该是 sourcemap

接下来这个 单元测试所以的点都是测试 sourcemap 的点，所以我们先不看了

那接着我们看看 baseCompile 函数是通过几个步骤生成的 code

baseCompile

javascript
export function baseCompile(
  template: string | RootNode,
  options: CompilerOptions = {}
): CodegenResult {
  const onError = options.onError || defaultOnError
  const isModuleMode = options.mode === 'module'
  /* istanbul ignore if */
  const ast = isString(template) ? baseParse(template, options) : template
  const [nodeTransforms, directiveTransforms] =
    getBaseTransformPreset(prefixIdentifiers)
  transform(
    ast,
    extend({}, options, {
      prefixIdentifiers,
      nodeTransforms: [
        ...nodeTransforms,
        ...(options.nodeTransforms || []) // user transforms
      ],
      directiveTransforms: extend(
        {},
        directiveTransforms,
        options.directiveTransforms || {} // user transforms
      )
    })
  )

  return generate(
    ast,
    extend({}, options, {
      prefixIdentifiers
    })
  )
}

好，最主要的就是三个部分

  1. 生成 ast - 通过 baseParse

  2. 调用 transform - 来处理 ast

  3. 使用 generate 生成 code

baseParse

那我们先看是如何生成ast的把

也就是baseParse，还是先看单元测试

下面的逻辑是只测试的 TextNode

javascript
    test('simple text', () => {
      const ast = baseParse('some text')
      const text = ast.children[0] as TextNode

      expect(text).toStrictEqual({
        type: NodeTypes.TEXT,
        content: 'some text',
        loc: {
          start: { offset: 0, line: 1, column: 1 },
          end: { offset: 9, line: 1, column: 10 },
          source: 'some text'
        }
      })
    })

可以看到 node 对象的关键的几个属性了

接着看看是如何解析出来的把

javascript
export function baseParse(
  content: string,
  options: ParserOptions = {}
): RootNode {
  const context = createParserContext(content, options)
  const start = getCursor(context)
  return createRoot(
    parseChildren(context, TextModes.DATA, []),
    getSelection(context, start)
  )
}

这里的重点是 context 是什么

继续去看 createParserContext

javascript
function createParserContext(
  content: string,
  rawOptions: ParserOptions
): ParserContext {
  const options = extend({}, defaultParserOptions)

  let key: keyof ParserOptions
  for (key in rawOptions) {
    // @ts-ignore
    options[key] =
      rawOptions[key] === undefined
        ? defaultParserOptions[key]
        : rawOptions[key]
  }
  return {
    options,
    column: 1,
    line: 1,
    offset: 0,
    originalSource: content,
    source: content,
    inPre: false,
    inVPre: false,
    onWarn: options.onWarn
  }
}

只是生成了一个配置对象

javascript
    {
      options: {
        delimiters: [ '{{', '}}' ],
        getNamespace: [Function: getNamespace],
        getTextMode: [Function: getTextMode],
        isVoidTag: [Function: NO],
        isPreTag: [Function: NO],
        isCustomElement: [Function: NO],
        decodeEntities: [Function: decodeEntities],
        onError: [Function: defaultOnError],
        onWarn: [Function: defaultOnWarn],
        comments: true
      },
      column: 1,
      line: 1,
      offset: 0,
      originalSource: 'some text',
      source: 'some text',
      inPre: false,
      inVPre: false,
      onWarn: [Function: defaultOnWarn]
    }

接着是调用了 getCursor

javascript
function getCursor(context: ParserContext): Position {
  const { column, line, offset } = context
  return { column, line, offset }
}

数据是来自 context 里面的

继续最后一个逻辑

javascript
  return createRoot(
    parseChildren(context, TextModes.DATA, []),
    getSelection(context, start)
  )

先看 parseChildren

javascript
function parseChildren(
  context: ParserContext,
  mode: TextModes,
  ancestors: ElementNode[]
): TemplateChildNode[] {
  const parent = last(ancestors)
  const ns = parent ? parent.ns : Namespaces.HTML
  const nodes: TemplateChildNode[] = []

  while (!isEnd(context, mode, ancestors)) {
    __TEST__ && assert(context.source.length > 0)
    const s = context.source
    let node: TemplateChildNode | TemplateChildNode[] | undefined = undefined

    if (mode === TextModes.DATA || mode === TextModes.RCDATA) {
      if (!context.inVPre && startsWith(s, context.options.delimiters[0])) {
        // '{{'
        node = parseInterpolation(context, mode)
      } else if (mode === TextModes.DATA && s[0] === '<') {

太多了，就不copy过来了。不过激动的是这里和我们之前去刷编译原理时候处理语法的时候逻辑是一致的，解析成功的话，那么就创建一个 node 节点对象

而且是用 nodes 把所有node对象都收集起来

至于解析的规则的话，就是按照 html 的规则来的

回头去刷编译原理就好了，这里去解析 html 的套路都是一样的

现在我们只需要知道返回一个处理完的 nodes就ok了

继续去看下一个逻辑点

getSelection

javascript
function getSelection(
  context: ParserContext,
  start: Position,
  end?: Position
): SourceLocation {
  end = end || getCursor(context)
  return {
    start,
    end,
    source: context.originalSource.slice(start.offset, end.offset)
  }
}

这里就是返回对应这段代码的信息的

createRoot

javascript
export function createRoot(
  children: TemplateChildNode[],
  loc = locStub
): RootNode {
  return {
    type: NodeTypes.ROOT,
    children,
    helpers: [],
    components: [],
    directives: [],
    hoists: [],
    imports: [],
    cached: 0,
    temps: 0,
    codegenNode: undefined,
    loc
  }
}

这里的 createRoot 就是直接创建一个 root 节点给外面就ok了

而关键的 children 就是通过parseChildren生成的 nodes。

transform

看看 transform 阶段是做了什么

javascript
  transform(
    ast,
    extend({}, options, {
      prefixIdentifiers,
      nodeTransforms: [
        ...nodeTransforms,
        ...(options.nodeTransforms || []) // user transforms
      ],
      directiveTransforms: extend(
        {},
        directiveTransforms,
        options.directiveTransforms || {} // user transforms
      )
    })
  )

是基于 ast 来做处理，第二个参数就是 transformOptions 了

javascript
export function transform(root: RootNode, options: TransformOptions) {
  const context = createTransformContext(root, options)
  traverseNode(root, context)
  if (options.hoistStatic) {
    hoistStatic(root, context)
  }
  if (!options.ssr) {
    createRootCodegen(root, context)
  }
  // finalize meta information
  root.helpers = [...context.helpers.keys()]
  root.components = [...context.components]
  root.directives = [...context.directives]
  root.imports = context.imports
  root.hoists = context.hoists
  root.temps = context.temps
  root.cached = context.cached

  if (__COMPAT__) {
    root.filters = [...context.filters!]
  }
}

还是先处理配置 context

看看长什么样子

javascript
 {
      selfName: null,
      prefixIdentifiers: false,
      hoistStatic: false,
      cacheHandlers: false,
      nodeTransforms: [ [Function: plugin] ],
      directiveTransforms: {},
      transformHoist: null,
      isBuiltInComponent: [Function: NOOP],
      isCustomElement: [Function: NOOP],
      expressionPlugins: [],
      scopeId: null,
      slotted: true,
      ssr: false,
      inSSR: false,
      ssrCssVars: '',
      bindingMetadata: {},
      inline: false,
      isTS: false,
      onError: [Function: defaultOnError],
      onWarn: [Function: defaultOnWarn],
      compatConfig: undefined,
      root: {
        type: 0,
        children: [ [Object] ],
        helpers: [],
        components: [],
        directives: [],
        hoists: [],
        imports: [],
        cached: 0,
        temps: 0,
        codegenNode: undefined,
        loc: {
          start: [Object],
          end: [Object],
          source: '<div>hello {{ world }}</div>'
        }
      },
      helpers: Map(0) {},
      components: Set(0) {},
      directives: Set(0) {},
      hoists: [],
      imports: [],
      constantCache: Map(0) {},
      temps: 0,
      cached: 0,
      identifiers: [Object: null prototype] {},
      scopes: { vFor: 0, vSlot: 0, vPre: 0, vOnce: 0 },
      parent: null,
      currentNode: {
        type: 0,
        children: [ [Object] ],
        helpers: [],
        components: [],
        directives: [],
        hoists: [],
        imports: [],
        cached: 0,
        temps: 0,
        codegenNode: undefined,
        loc: {
          start: [Object],
          end: [Object],
          source: '<div>hello {{ world }}</div>'
        }
      },
      childIndex: 0,
      inVOnce: false,
      helper: [Function: helper],
      removeHelper: [Function: removeHelper],
      helperString: [Function: helperString],
      replaceNode: [Function: replaceNode],
      removeNode: [Function: removeNode],
      onNodeRemoved: [Function: onNodeRemoved],
      addIdentifiers: [Function: addIdentifiers],
      removeIdentifiers: [Function: removeIdentifiers],
      hoist: [Function: hoist],
      cache: [Function: cache],
      filters: Set(0) {}
    }

这里的好多属性看起来都是 vue 特有的

第二步的时候就是 调用 traverseNode

后面的逻辑是处理一些特殊key 的

javascript
  root.helpers = [...context.helpers.keys()]
  root.components = [...context.components]
  root.directives = [...context.directives]
  root.imports = context.imports
  root.hoists = context.hoists
  root.temps = context.temps
  root.cached = context.cached

这里是把一些额外的方法给到了 root 上面，而root 是 AST 的root node

traverseNode

在看这个函数之前，先找个测试看看

javascript
  test('context state', () => {
    const ast = baseParse(`<div>hello {{ world }}</div>`)

    // manually store call arguments because context is mutable and shared
    // across calls
    const calls: any[] = []
    const plugin: NodeTransform = (node, context) => {
      calls.push([node, { ...context }])
    }

    transform(ast, {
      nodeTransforms: [plugin]
    })

    const div = ast.children[0] as ElementNode
    expect(calls.length).toBe(4)
    expect(calls[0]).toMatchObject([
      ast,
      {
        parent: null,
        currentNode: ast
      }
    ])
    expect(calls[1]).toMatchObject([
      div,
      {
        parent: ast,
        currentNode: div
      }
    ])
    expect(calls[2]).toMatchObject([
      div.children[0],
      {
        parent: div,
        currentNode: div.children[0]
      }
    ])
    expect(calls[3]).toMatchObject([
      div.children[1],
      {
        parent: div,
        currentNode: div.children[1]
      }
    ])
  })

通过这个测试可以知道，transform 是在本身的 AST 的基础上直接修改数据的

而这里的执行模式应该和 babel 的 plugin 的形式也差不多，通过 visit 的处理方式来调用

上面的 nodeTransforms:[plugin] 就是处理方式，看起来是当所有的node调用的时候，就会执行这个 nodeTransforms 里面给的函数

在看看第二个测试

javascript
  test('context.replaceNode', () => {
    const ast = baseParse(`<div/><span/>`)
    const plugin: NodeTransform = (node, context) => {
      if (node.type === NodeTypes.ELEMENT && node.tag === 'div') {
        // change the node to <p>
        context.replaceNode(
          Object.assign({}, node, {
            tag: 'p',
            children: [
              {
                type: NodeTypes.TEXT,
                content: 'hello',
                isEmpty: false
              }
            ]
          })
        )
      }
    }
    const spy = jest.fn(plugin)
    transform(ast, {
      nodeTransforms: [spy]
    })

    expect(ast.children.length).toBe(2)
    const newElement = ast.children[0] as ElementNode
    expect(newElement.tag).toBe('p')
    expect(spy).toHaveBeenCalledTimes(4)
    // should traverse the children of replaced node
    expect(spy.mock.calls[2][0]).toBe(newElement.children[0])
    // should traverse the node after the replaced node
    expect(spy.mock.calls[3][0]).toBe(ast.children[1])
  })

在 plugin 的实现里面可以看到就是通过替换node来达到修改代码的效果

而context 是什么？哦，看起来 context 是用来处理 node 的

traverseNode

使用的基本逻辑明白了 那接着看看 traverseNode 内部是如何实现的把

javascript
export function traverseNode(
  node: RootNode | TemplateChildNode,
  context: TransformContext
) {
  context.currentNode = node
  // apply transform plugins
  const { nodeTransforms } = context
  const exitFns = []
  for (let i = 0; i < nodeTransforms.length; i++) {
    const onExit = nodeTransforms[i](node, context)
    if (onExit) {
      if (isArray(onExit)) {
        exitFns.push(...onExit)
      } else {
        exitFns.push(onExit)
      }
    }
    if (!context.currentNode) {
      // node was removed
      return
    } else {
      // node may have been replaced
      node = context.currentNode
    }
  }

  switch (node.type) {
    case NodeTypes.COMMENT:
      if (!context.ssr) {
        // inject import for the Comment symbol, which is needed for creating
        // comment nodes with `createVNode`
        context.helper(CREATE_COMMENT)
      }
      break
    case NodeTypes.INTERPOLATION:
      // no need to traverse, but we need to inject toString helper
      if (!context.ssr) {
        context.helper(TO_DISPLAY_STRING)
      }
      break

    // for container types, further traverse downwards
    case NodeTypes.IF:
      for (let i = 0; i < node.branches.length; i++) {
        traverseNode(node.branches[i], context)
      }
      break
    case NodeTypes.IF_BRANCH:
    case NodeTypes.FOR:
    case NodeTypes.ELEMENT:
    case NodeTypes.ROOT:
      traverseChildren(node, context)
      break
  }

  // exit transforms
  context.currentNode = node
  let i = exitFns.length
  while (i--) {
    exitFns[i]()
  }
}

这里的 context 就是通过 createTransformContext 生成的，里面有好多方法可以处理 node

第一步是先调用用户通过 config 注入的 nodeTransforms 里面的函数，也就是单测里面给的 plugin 函数

参数就是把 node 和 context 给到，所以用户可以在 plugin 里面通过 context 提供的方法来处理 node

这里的细节是， plugin 是可以返回一个函数的，这个函数就做 onExit

接着会基于 node 的类型做不同的处理

javascript
switch (node.type) {
    case NodeTypes.COMMENT:
      if (!context.ssr) {
        // inject import for the Comment symbol, which is needed for creating
        // comment nodes with `createVNode`
        context.helper(CREATE_COMMENT)
      }
      break
    case NodeTypes.INTERPOLATION:
      // no need to traverse, but we need to inject toString helper
      if (!context.ssr) {
        context.helper(TO_DISPLAY_STRING)
      }
      break

    // for container types, further traverse downwards
    case NodeTypes.IF:
      for (let i = 0; i < node.branches.length; i++) {
        traverseNode(node.branches[i], context)
      }
      break
    case NodeTypes.IF_BRANCH:
    case NodeTypes.FOR:
    case NodeTypes.ELEMENT:
    case NodeTypes.ROOT:
      traverseChildren(node, context)
      break
  }

  • NodeTypes.COMMENT → context.helper(CREATE_COMMENT)

  • NodeTypes.INTERPOLATION → context.helper(TO_DISPLAY_STRING)

  • NodeTypes.IF → traverseNode(node.branches[i], context)

  • NodeTypes.IF_BRANCH || NodeTypes.FOR || NodeTypes.ELEMENT || NodeTypes.ROOT:

    traverseChildren(node, context)

这个处理完成后在统一的调用 exitFn

javascript
  // exit transforms
  context.currentNode = node
  let i = exitFns.length
  while (i--) {
    exitFns[i]()
  }

这里应该是方便让用户做一些清理逻辑

createTransformContext - context

继续来分析一下 context

他里面有几个关键的方法

javascript
  helper(name) {
 
    },
    removeHelper(name) {
  
    },
    helperString(name) {
    
    },
    replaceNode(node) {
    },
    removeNode(node) {
    onNodeRemoved: () => {},
    addIdentifiers(exp) {
     
    },
    removeIdentifiers(exp) {
      
    },
    hoist(exp) {
     
    },
    cache(exp, isVNode = false) {
      
    }

先来看看 helper

javascript
    helper(name) {
      const count = context.helpers.get(name) || 0
      context.helpers.set(name, count + 1)
      return name
    },

逻辑是加一个 count ，那么是用在哪里的呢？

没找到 继续看看 removeHelper

javascript
    removeHelper(name) {
      const count = context.helpers.get(name)
      if (count) {
        const currentCount = count - 1
        if (!currentCount) {
          context.helpers.delete(name)
        } else {
          context.helpers.set(name, currentCount)
        }
      }
    },

和 helper 是对应的，这里是删除一个 count

在看 helperString

javascript
    helperString(name) {
    return `_${helperNameMap[context.helper(name)]}`
    },

这里的重点是 helperNameMap ，而 context.helper(name ) 是基于 name 计数了一下，然后把 name 返回。

那看看 helperNameMap

javascript
export const helperNameMap: any = {
  [FRAGMENT]: `Fragment`,
  [TELEPORT]: `Teleport`,
  [SUSPENSE]: `Suspense`,
  [KEEP_ALIVE]: `KeepAlive`,
  [BASE_TRANSITION]: `BaseTransition`,
  [OPEN_BLOCK]: `openBlock`,
  [CREATE_BLOCK]: `createBlock`,
  [CREATE_ELEMENT_BLOCK]: `createElementBlock`,
  [CREATE_VNODE]: `createVNode`,
  [CREATE_ELEMENT_VNODE]: `createElementVNode`,
  [CREATE_COMMENT]: `createCommentVNode`,

太多了，截取了一部分，可以看到，这里存储的都是对应的处理函数，也就是所谓的 helper

那可以说是 helperString 就是返回对应 helper 的名称

继续看replaceNod

Source: cuixiaorui/mini-vue

View original on GitHubView discussion on GitHub