SwiftTreeSitter

main

Star
Swift API for the tree-sitter incremental parsing system
wabiverse/SwiftTreeSitter

Build Status Platforms Documentation Discord

SwiftTreeSitter

Swift API for the tree-sitter incremental parsing system.

  • Close to full coverage of the C API
  • Swift/Foundation types where possible
  • Standard query result mapping for highlights and injections
  • Query predicate/directive support via ResolvingQueryMatchSequence
  • Nested language support
  • Swift concurrency support where possible

Structure

This project is actually split into two parts: SwiftTreeSitter and SwiftTreeSitterLayer.

The SwiftTreeSitter target is a close match to the C runtime API. It adds only a few additional types to help support querying. It is fairly low-level, and there will be significant work to use it in a real project.

SwiftTreeSitterLayer is an abstraction built on top of SwiftTreeSitter. It supports documents with nested languages and transparent querying across those nestings. It also supports asynchronous language resolution. While still low-level, SwiftTreeSitterLayer is easier to work with while also supporting more features.

And yet there's more! If you are looking a higher-level system for syntax highlighting and other syntactic operations, you might want to have a look at Neon. It is much easier to integrate with a text system, and has lots of additional performance-related features.

Integration

dependencies: [
    .package(url: "https://github.com/ChimeHQ/SwiftTreeSitter")
],
targets: [
    .target(
        name: "MySwiftTreeSitterTarget",
        dependencies: ["SwiftTreeSitter"]
    ),
    .target(
        name: "MySwiftTreeSitterLayerTarget",
        dependencies: [
            .product(name: "SwiftTreeSitterLayer", package: "SwiftTreeSitter"),
        ]
    ),
]

Query Conflicts

SwiftTreeSitter does its best to resolve poor/incorrect query constructs, which are surprisingly common.

When using injections, child query ranges are automatically expanded using parent matches. This handles cases where a parent has queries that overlap with children in conflicting ways. Without expansion, it is possible to construct queries that fall within children ranges but produce on parent matches.

All matches are sorted by:

  • depth
  • location in content
  • specificity of match label (more components => more specific)
  • occurrence in the query source

Even with these, it is possible to produce queries that will result in "incorrect" behavior that are either ambiguous or undefined in the query definition.

Highlighting

A very common use of tree-sitter is to do syntax highlighting. It is possible to use this library directly, especially if your source text does not change. Here's a little example that sets everything up with a SPM-bundled language.

First, check out how it works with SwiftTreeSitterLayer. It's complex, but does a lot for you.

// LanguageConfiguration takes care of finding and loading queries in SPM-created bundles.
let markdownConfig = try LanguageConfiguration(tree_sitter_markdown(), name: "Markdown")
let markdownInlineConfig = try LanguageConfiguration(
    tree_sitter_markdown_inline(),
    name: "MarkdownInline",
    bundleName: "TreeSitterMarkdown_TreeSitterMarkdownInline"
)
let swiftConfig = try LanguageConfiguration(tree_sitter_swift(), name: "Swift")

// Unfortunately, injections do not use standardized language names, and can even be content-dependent. Your system must do this mapping.
let config = LanguageLayer.Configuration(
    languageProvider: {
        name in
        switch name {
        case "markdown":
            return markdownConfig
        case "markdown_inline":
            return markdownInlineConfig
        case "swift":
            return swiftConfig
        default:
            return nil
        }
    }
)

let rootLayer = try LanguageLayer(languageConfig: markdownConfig, configuration: config)

let source = """
# this is markdown

```swift
func main(a: Int) {
}

also markdown

let value = "abc"

"""

rootLayer.replaceContent(with: source)

let fullRange = NSRange(source.startIndex..<source.endIndex, in: source)

let textProvider = source.predicateTextProvider let highlights = try rootLayer.highlights(in: fullRange, provider: textProvider)

for namedRange in highlights { print("(namedRange.name): (namedRange.range)") }


You can also use SwiftTreeSitter directly:

```swift
let swiftConfig = try LanguageConfiguration(tree_sitter_swift(), name: "Swift")

let parser = Parser()
try parser.setLanguage(swiftConfig.language)

let source = """
func main() {}
"""
let tree = parser.parse(source)!

let query = swiftConfig.queries[.highlights]!

let cursor = query.execute(in: tree)
let highlights = cursor
    .resolve(with: .init(string: source))
    .highlights()

for namedRange in highlights {
    print("range: ", namedRange)
}

Language Parsers

Tree-sitter language parsers are separate projects, and you'll probably need at least one. More details are available in the documentation. How they can be installed an incorporated varies.

Here's a list of parsers that support SPM. Since you're here, you might find that convenient. And the LanguageConfiguration type supports loading bundled queries directly.

Parser Make SPM Official Repo
Bash ✅ ✅
C ✅ ✅
C++ ✅ ✅
C# ✅ ✅
Clojure ✅
CSS ✅ ✅ ✅
Dockerfile ✅ ✅ ✅
Diff ✅ ✅
Elixir ✅ ✅ ✅
Elm ✅ ✅
Go ✅ ✅ ✅
GoMod ✅ ✅ ✅
GoWork ✅
Haskell ✅ ✅
HCL ✅ ✅
HTML ✅ ✅
Java ✅ ✅ ✅
Javascript ✅ ✅
JSON ✅ ✅ ✅
JSDoc ✅ ✅
Julia ✅ ✅
Kotlin ✅
Latex ✅ ✅ ✅
Lua ✅ ✅
Markdown ✅ ✅
OCaml ✅ ✅
Perl ✅ ✅
PHP ✅ ✅ ✅
Pkl ✅ ✅
Python ✅ ✅
Ruby ✅ ✅ ✅
Rust ✅ ✅
Scala ✅ ✅
SQL ✅ ✅
SSH ✅ ✅
Swift ✅ ✅ ✅
TOML ✅
Tree-sitter query language ✅ ✅
Typescript ✅ ✅
Verilog ✅ ✅
YAML ✅
Zig ✅ ✅ ✅

Contributing and Collaboration

I would love to hear from you! Issues or pull requests work great. A Discord server is also available for live help, but I have a strong bias towards answering in the form of documentation.

I prefer collaboration, and would love to find ways to work together if you have a similar project.

I prefer indentation with tabs for improved accessibility. But, I'd rather you use the system you want and make a PR than hesitate because of whitespace.

By participating in this project you agree to abide by the Contributor Code of Conduct.

Description

  • Swift Tools 5.9.0
View More Packages from this Author

Dependencies

Last updated: Thu Oct 17 2024 22:20:25 GMT-0900 (Hawaii-Aleutian Daylight Time)