Release 2.0

After 2 years, a new release finally arrived!

v2.0 is a major release, which contains some breaking changes.

Use parameters packs in HTMLComponentBuilder, removes a lot of boilerplate code!
Requires swift 5.9 or above.
Without depends on any html parser.
Now also available on Linux!

Type construction changes

There is some breaking changes in the type construction:

HTML {
    Capture
    Local {
        Capture
        Capture
    }
}

previously the compiler will construct this output as (C, C, C),
but in version 2.0 this will be constructed as (C, (C, C)).

Build your own parser

HTMLParserBuilder now doesn't rely on any html parser, so you can chose any html parser you want to use, as long as it conforms to the Document and Element protocol.

For example, you can use SwiftSoup as the html parser, example for conformance to the Document and Element protocol is available in Tests/HTMLParserBuilderTests/SwiftSoup+HTMLParserBuilder.swift.

HTMLParserBuilder

A result builder that build HTML parser and transform HTML elements to strongly-typed result, inspired by RegexBuilder.

Note: CaptureTransform.swift, TypeConstruction.swift are copied from apple/swift-experimental-string-processing.

HTMLParserBuilder
- Installation
  - Requirement
- Introduction
- Usage
  - Bring your own parser
  - Parsing
  - HTML
  - Capture
  - TryCapture
  - CaptureAll
  - Local
  - LateInit
  - Wrap Up
- Advanced use case

Installation

Requirement

Swift 5.9
macOS 10.15
iOS 13.0
tvOS 13.0
watchOS 6.0

dependencies: [
    // ...
    .package(name: "HTMLParserBuilder", url: "https://github.com/danny1113/html-parser-builder.git", from: "2.0.0"),
]

Introduction

Parsing HTML can be complicated, for example you want to parse the simple html below:

<h1 id="hello">hello, world</h1>

<div id="group">
    <h1>INSIDE GROUP h1</h1>
    <h2>INSIDE GROUP h2</h2>
</div>

Existing HTML parsing library have these downside:

Name every captured element
It can be more complex as the element you want to capture become more and more
Error handling can be hard

let htmlString = "<html>...</html>"
let doc: any Document = HTMLDocument(string: htmlString)
let first = doc.querySelector("#hello")?.textContent

let group = doc.querySelector("#group")
let second = group?.querySelector("h1")?.textContent
let third = group?.querySelector("h2")?.textContent

if  let first = first,
    let second = second,
    let third = third {
    
    // ...
} else {
    // ...
}

HTMLParserBuilder comes with some really great advantages:

Strongly-typed capture result
Structrued syntax
Composible API
Support for async await
Error handling built in

You can construct your parser which reflect your original HTML structure:

let capture = HTML {
    TryCapture("#hello") { (element: any Element?) -> String? in
        return element?.textContent
    } // => HTML<String?>
    
    Local("#group") {
        Capture("h1", transform: \.textContent) // => HTML<String>
        Capture("h2", transform: \.textContent) // => HTML<String>
    } // => HTML<(String, String)>
    
} // => HTML<(String?, (String, String))>


let htmlString = "<html>...</html>"
let doc: any Document = HTMLDocument(string: htmlString)

let output = try doc.parse(capture)
// => (String?, (String, String))
// output: (Optional("hello, world"), ("INSIDE GROUP h1", "INSIDE GROUP h2"))

Note: You can now compose up to 10 components inside the builder, but you can group your captures inside Local as a workaround.

Usage

Bring your own parser

HTMLParserBuilder doesn't rely on any html parser, so you can chose any html parser you want to use, as long as it conforms to the Document and Element protocol.

dependencies: [
    // ...
    .package(url: "https://github.com/scinfu/SwiftSoup.git", from: "2.6.0"),
    .package(name: "HTMLParserBuilder", url: "https://github.com/danny1113/html-parser-builder.git", from: "2.0.0"),
],
targets: [
    .target(name: "YourTarget", dependencies: ["SwiftSoup", "HTMLParserBuilder"]),
]

Parsing

HTMLParserBuilder provides 2 functions for parsing:

public func parse<Output>(_ html: HTML<Output>) throws -> Output
public func parse<Output>(_ html: HTML<Output>) async throws -> Output

Note: You can choose the async version for even better performance, since it use structured concurrency to parallelize child tasks.

HTML

You can construct your parser inside HTML, it can also transform to other data type.

struct Group {
    let h1: String
    let h2: String
}

let capture = HTML {
    Capture("#group h1", transform: \.textContent) // => HTML<String>
    Capture("#group h2", transform: \.textContent) // => HTML<String>
    
} transform: { (output: (String, String)) -> Group in
    return Group(
        h1: output.0,
        h2: output.1
    )
} // => HTML<Group>

Capture

Using Capture is the same as querySelector, you pass in CSS selector to find the HTML element, and you can transform it to any other type you want:

innerHTML
textContent
attributes
...

Note: If Capture can't find the HTML element that match the selector, it will throw an error cause the whole parse fail, for failable capture, see TryCapture.

You can use this API with various declaration that is most suitable for you:

Capture("#hello", transform: \.textContent)
Capture("#hello") { $0.textContent }
Capture("#hello") { (e: any Element) -> String in
    return e.textContent
}

TryCapture

TryCapture is a litte different from Capture, it also calls querySelector to find the HTML element, but it returns an optional HTML element.

For this example, it will produce the result type of String?, and the result will be nil when the HTML element can't be found.

TryCapture("#hello") { (e: (any Element)?) -> String? in
    return e?.innerHTML
}

CaptureAll

Using CaptureAll is the same as querySelectorAll, you pass in CSS selector to find all HTML elements that match the selector, and you can transform it to any other type you want:

You can use this API with various declaration that is most suitable for you:

CaptureAll("h1") { $0.map(\.textContent) }
CaptureAll("h1") { (e: [any Element]) -> [String] in
    return e.map(\.textContent)
}

You can also capture other elements inside and transform to other type:

<div class="group">
    <h1>Group 1</h1>
</div>
<div class="group">
    <h1>Group 2</h1>
</div>

CaptureAll("div.group") { (elements: [any Element]) -> [String] in
    return elements.compactMap { e in
        return e.querySelector("h1")?.textContent
    }
}
// => [String]
// output: ["Group 1", "Group 2"]

Local

Local will find a HTML element that match the selector, and all the captures inside will find its element based on the element found by Local, this is useful when you just want to capture element that is inside the local group.

Just like HTML, Local can also transform captured result to other data type by adding transform:

struct Group {
    let h1: String
    let h2: String
}

Local("#group") {
    Capture("h1", transform: \.textContent) // => HTML<String>
    Capture("h2", transform: \.textContent) // => HTML<String>
} transform: { (output: (String, String)) -> Group in
    return Group(
        h1: output.0,
        h2: output.1
    )
} // => Group

Note: If Local can't find the HTML element that match the selector, it will throw an error cause the whole parse fail, you can use TryCapture as alternative.

LateInit

This library also comes with a handy property wrapper: LateInit, which can delay the initialization until the first time you access it.

struct Container {
    @LateInit var capture = HTML {
        Capture("h1", transform: \.textContent)
    }
}

// it needs to be `var` to perform late initialization
var container = Container()
let output = doc.parse(container.capture)
// ...

Wrap Up

API	Use Case
Capture	Throws error when element can't be captured
TryCapture	Returns `nil` when element can't be captured
CaptureAll	Capture all elements match the selector
Local	Capture elements in the local scope
LateInit	Delay the initialization to first time you access it

Advanced use case

Pass HTMLComponent into another
Transform to custom data structure before parasing

struct Group {
    let h1: String
    let h2: String
}

//       |--------------------------------------------------------------|
let groupCapture = HTML {                                            // |
    Local("#group") {                                                // |
        Capture("h1", transform: \.textContent) // => HTML<String>   // |
        Capture("h2", transform: \.textContent) // => HTML<String>   // |
    } // => HTML<(String, String)>                                   // |
                                                                     // |
} transform: { output -> Group in                                    // |
    return Group(                                                    // |
        h1: output.0,                                                // |
        h2: output.1                                                 // |
    )                                                                // |
} // => HTML<Group>                                                  // |
                                                                     // |
let capture = HTML {                                                 // |
    TryCapture("#hello") { (element: (any Element)?) -> String? in   // |
        return element?.textContent                                  // |
    } // => HTML<String?>                                            // |
                                                                     // |
    groupCapture // => HTML<Group> -------------------------------------|
    
} // => HTML<(String?, Group)>


let htmlString = "<html>...</html>"
let doc: any Document = HTMLDocument(string: htmlString)

let output = try doc.parse(capture)
// => (String?, Group)

HTMLParserBuilder

2.0.0

What's New

2.0.0

2024-06-11T09:04:02Z

Release 2.0

Type construction changes

Build your own parser

HTMLParserBuilder

Installation

Requirement

Introduction

Usage

Bring your own parser

Parsing

HTML

Capture

TryCapture

CaptureAll

Local

LateInit

Wrap Up

Advanced use case

Description

Dependencies