A11 (C++ runtime)
Native C++ implementation of the A11 streaming action runtime
Loading...
Searching...
No Matches
token.h
Go to the documentation of this file.
1/*
2 * Copyright 2026 The A11 Authors
3 *
4 * Licensed under the Apache License, Version 2.0 (the "License");
5 * you may not use this file except in compliance with the License.
6 * You may obtain a copy of the License at
7 *
8 * http://www.apache.org/licenses/LICENSE-2.0
9 *
10 * Unless required by applicable law or agreed to in writing, software
11 * distributed under the License is distributed on an "AS IS" BASIS,
12 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 * See the License for the specific language governing permissions and
14 * limitations under the License.
15 */
16
17#ifndef A11_FLOW_TOKEN_H_
18#define A11_FLOW_TOKEN_H_
19
20#include <cstddef>
21#include <string>
22#include <string_view>
23
24#include <absl/time/time.h>
25
26namespace a11::flow {
27
36enum class TokenKind {
42 kString,
44 kNumber,
48 kWord,
49
50 kDot,
52 kRange,
54 kSpread,
55 kArrow,
56 kCarry,
57 kEqual,
60 kLess,
64 kPlus,
65 kMinus,
66 kPipe,
67 kColon,
68 kComma,
75
78 kBad,
80 kEnd,
81};
82
87struct Token {
90 std::string_view text;
91 size_t start = 0;
92 size_t end = 0;
94 int line = 1;
95 int column = 1;
96
98 std::string string_value;
101 double number = 0.0;
102 bool is_integer = false;
104 absl::Duration duration;
105
106 [[nodiscard]] bool IsWord() const { return kind == TokenKind::kWord; }
107
110 [[nodiscard]] bool EndsStatement() const {
113 }
114};
115
120std::string_view KindName(TokenKind kind);
121
123TokenKind KindFromName(std::string_view name);
124
125} // namespace a11::flow
126
127#endif // A11_FLOW_TOKEN_H_
TokenKind kind
Definition format.cc:913
std::string name
The name and its colon, which travel together because they always do.
Definition format.cc:49
Definition boundary.cc:32
TokenKind KindFromName(std::string_view name)
The kind a name from [KindName] refers to, or kBad if it is not one.
@ kBad
Something the language has no meaning for.
std::string_view KindName(TokenKind kind)
The name of a kind, as a11.flow.lexer spells it.
@ kCarry
What a repeat carries.
TokenKind
What kind of thing a token is.
Definition token.h:36
@ kRange
.. – the range between two bounds, either of which may be left out.
@ kEnd
One past the last token, so lookahead never runs off the end.
@ kWord
A bare word. What it means is the grammar's business, not the lexer's.
@ kNewline
The end of a statement. One per run of blank lines, never leading.
@ kSpread
... or ... – everything the thing after it holds, spread in here.
One token: what it is, where it is, and what it holds.
Definition token.h:87
size_t start
Definition token.h:91
int line
1-based, as a11.flow.lexer has always reported them.
Definition token.h:94
bool IsWord() const
Definition token.h:106
int column
Definition token.h:95
bool is_integer
Definition token.h:102
std::string_view text
The source slice, exactly as written.
Definition token.h:90
bool EndsStatement() const
Whether a line break, a } or the end of the file is here: the three ways a statement can end.
Definition token.h:110
size_t end
Definition token.h:92
std::string string_value
A kString's value, with escapes resolved.
Definition token.h:98
double number
A kNumber's value.
Definition token.h:101
absl::Duration duration
A kDuration's value.
Definition token.h:104
TokenKind kind
Definition token.h:88