Actually commit api-client-php's vendor/ - it was silently gitignored
Docker-Images bauen und veröffentlichen / build (, omsorgCore/Dockerfile, omsorgcore) (push) Successful in 3s
Docker-Images bauen und veröffentlichen / build (, omsorgWeb/Dockerfile, omsorgweb) (push) Successful in 6s
Docker-Images bauen und veröffentlichen / build (, omsorgapp/Dockerfile, omsorgapp) (push) Successful in 4s
Docker-Images bauen und veröffentlichen / build (, omsorgCore/Dockerfile, omsorgcore) (push) Successful in 3s
Docker-Images bauen und veröffentlichen / build (, omsorgWeb/Dockerfile, omsorgweb) (push) Successful in 6s
Docker-Images bauen und veröffentlichen / build (, omsorgapp/Dockerfile, omsorgapp) (push) Successful in 4s
omsorgCore/CLAUDE.md documents vendor/ as committed so the app can run without a composer install step, but the generator's own .gitignore (api-client-php/.gitignore) excludes /vendor/ - it only ever existed untracked on disk locally, which is why the Docker build worked for me but mitarbeiter-app crashed at runtime on a real (fresh-checkout) deploy: "Failed opening required '.../api-client-php/vendor/autoload.php'". Verified by building from a git-archive-simulated fresh checkout with this fix applied - mitarbeiter-app's login page now renders without the fatal error. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Felix Kemmler
co-authored by
Claude Sonnet 5
Claude Sonnet 5
parent
c8a514af9b
commit
dca0349e8c
+2
@@ -0,0 +1,2 @@
|
||||
github: clue
|
||||
custom: https://clue.engineering/support
|
||||
+75
@@ -0,0 +1,75 @@
|
||||
# Changelog
|
||||
|
||||
## 1.3.0 (2022-12-23)
|
||||
|
||||
* Feature: Add support for PHP 8.1 and PHP 8.2.
|
||||
(#31 by @clue and #30 by @SimonFring)
|
||||
|
||||
* Feature: Check type of incoming `data` before trying to decode NDJSON.
|
||||
(#29 by @SimonFrings)
|
||||
|
||||
* Improve documentation and examples and update to new [default loop](https://reactphp.org/event-loop/#loop).
|
||||
(#26 by @clue, #27 by @SimonFrings and #25 by @PaulRotmann)
|
||||
|
||||
* Improve test suite, report failed assertions and ensure 100% code coverage.
|
||||
(#32 and #33 by @clue and #28 by @SimonFrings)
|
||||
|
||||
## 1.2.0 (2020-12-09)
|
||||
|
||||
* Improve test suite and add `.gitattributes` to exclude dev files from exports.
|
||||
Add PHP 8 support, update to PHPUnit 9 and simplify test setup.
|
||||
(#18 by @clue and #19, #22 and #23 by @SimonFrings)
|
||||
|
||||
## 1.1.0 (2020-02-04)
|
||||
|
||||
* Feature: Improve error reporting and add parsing error message to Exception and
|
||||
ignore `JSON_THROW_ON_ERROR` option (available as of PHP 7.3).
|
||||
(#14 by @clue)
|
||||
|
||||
* Feature: Add bechmarking script and import all global function references.
|
||||
(#16 by @clue)
|
||||
|
||||
* Improve documentation and add NDJSON format description and
|
||||
add support / sponsorship info.
|
||||
(#12 and #17 by @clue)
|
||||
|
||||
* Improve test suite to run tests on PHP 7.4 and simplify test matrix and
|
||||
apply minor code style adjustments to make phpstan happy.
|
||||
(#13 and #15 by @clue)
|
||||
|
||||
## 1.0.0 (2018-05-17)
|
||||
|
||||
* First stable release, now following SemVer
|
||||
|
||||
* Improve documentation and usage examples
|
||||
|
||||
> Contains no other changes, so it's actually fully compatible with the v0.1.2 release.
|
||||
|
||||
## 0.1.2 (2018-05-11)
|
||||
|
||||
* Feature: Limit buffer size to 64 KiB by default.
|
||||
(#10 by @clue)
|
||||
|
||||
* Feature: Forward compatiblity with EventLoop v0.5 and upcoming v1.0.
|
||||
(#8 by @clue)
|
||||
|
||||
* Fix: Return bool `false` if encoding fails due to invalid value to pause source.
|
||||
(#9 by @clue)
|
||||
|
||||
* Improve test suite by supporting PHPUnit v6 and test against legacy PHP 5.3 through PHP 7.2.
|
||||
(#7 by @clue)
|
||||
|
||||
* Update project homepage.
|
||||
(#11 by @clue)
|
||||
|
||||
## 0.1.1 (2017-05-22)
|
||||
|
||||
* Feature: Forward compatibility with Stream v0.7, v0.6, v0.5 and upcoming v1.0 (while keeping BC)
|
||||
(#6 by @thklein)
|
||||
|
||||
* Improved test suite by adding PHPUnit to `require-dev`
|
||||
(#5 by @thklein)
|
||||
|
||||
## 0.1.0 (2016-11-24)
|
||||
|
||||
* First tagged release
|
||||
@@ -0,0 +1,21 @@
|
||||
The MIT License (MIT)
|
||||
|
||||
Copyright (c) 2016 Christian Lück
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is furnished
|
||||
to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
@@ -0,0 +1,365 @@
|
||||
# clue/reactphp-ndjson
|
||||
|
||||
[](https://github.com/clue/reactphp-ndjson/actions)
|
||||
[](https://packagist.org/packages/clue/ndjson-react)
|
||||
[](#tests)
|
||||
|
||||
Streaming newline-delimited JSON ([NDJSON](http://ndjson.org/)) parser and encoder for [ReactPHP](https://reactphp.org/).
|
||||
|
||||
[NDJSON](http://ndjson.org/) can be used to store multiple JSON records in a
|
||||
file to store any kind of (uniform) structured data, such as a list of user
|
||||
objects or log entries. It uses a simple newline character between each
|
||||
individual record and as such can be both used for efficient persistence and
|
||||
simple append-style operations. This also allows it to be used in a streaming
|
||||
context, such as a simple inter-process communication (IPC) protocol or for a
|
||||
remote procedure call (RPC) mechanism. This library provides a simple
|
||||
streaming API to process very large NDJSON files with thousands or even millions
|
||||
of rows efficiently without having to load the whole file into memory at once.
|
||||
|
||||
* **Standard interfaces** -
|
||||
Allows easy integration with existing higher-level components by implementing
|
||||
ReactPHP's standard streaming interfaces.
|
||||
* **Lightweight, SOLID design** -
|
||||
Provides a thin abstraction that is [*just good enough*](https://en.wikipedia.org/wiki/Principle_of_good_enough)
|
||||
and does not get in your way.
|
||||
Builds on top of well-tested components and well-established concepts instead of reinventing the wheel.
|
||||
* **Good test coverage** -
|
||||
Comes with an [automated tests suite](#tests) and is regularly tested in the *real world*.
|
||||
|
||||
**Table of contents**
|
||||
|
||||
* [Support us](#support-us)
|
||||
* [NDJSON format](#ndjson-format)
|
||||
* [Usage](#usage)
|
||||
* [Decoder](#decoder)
|
||||
* [Encoder](#encoder)
|
||||
* [Install](#install)
|
||||
* [Tests](#tests)
|
||||
* [License](#license)
|
||||
* [More](#more)
|
||||
|
||||
## Support us
|
||||
|
||||
We invest a lot of time developing, maintaining, and updating our awesome
|
||||
open-source projects. You can help us sustain this high-quality of our work by
|
||||
[becoming a sponsor on GitHub](https://github.com/sponsors/clue). Sponsors get
|
||||
numerous benefits in return, see our [sponsoring page](https://github.com/sponsors/clue)
|
||||
for details.
|
||||
|
||||
Let's take these projects to the next level together! 🚀
|
||||
|
||||
## NDJSON format
|
||||
|
||||
NDJSON ("Newline-Delimited JSON" or sometimes referred to as "JSON lines") is a
|
||||
very simple text-based format for storing a large number of records, such as a
|
||||
list of user records or log entries.
|
||||
|
||||
```JSON
|
||||
{"name":"Alice","age":30,"comment":"Yes, I like cheese"}
|
||||
{"name":"Bob","age":50,"comment":"Hello\nWorld!"}
|
||||
```
|
||||
|
||||
If you understand JSON and you're now looking at this newline-delimited JSON for
|
||||
the first time, you should already know everything you need to know to
|
||||
understand NDJSON: As the name implies, this format essentially consists of
|
||||
individual lines where each individual line is any valid JSON text and each line
|
||||
is delimited with a newline character.
|
||||
|
||||
This example uses a list of user objects where each user has some arbitrary
|
||||
properties. This can easily be adjusted for many different use cases, such as
|
||||
storing for example products instead of users, assigning additional properties
|
||||
or having a significantly larger number of records. You can edit NDJSON files in
|
||||
any text editor or use them in a streaming context where individual records
|
||||
should be processed. Unlike normal JSON files, adding a new log entry to this
|
||||
NDJSON file does not require modification of this file's structure (note there's
|
||||
no "outer array" to be modified). This makes it a perfect fit for a streaming
|
||||
context, for line-oriented CLI tools (such as `grep` and others) or for a logging
|
||||
context where you want to append records at a later time. Additionally, this
|
||||
also allows it to be used in a streaming context, such as a simple inter-process
|
||||
communication (IPC) protocol or for a remote procedure call (RPC) mechanism.
|
||||
|
||||
The newline character at the end of each line allows for some really simple
|
||||
*framing* (detecting individual records). While each individual line is valid
|
||||
JSON, the complete file as a whole is technically no longer valid JSON, because
|
||||
it contains multiple JSON texts. This implies that for example calling PHP's
|
||||
`json_decode()` on this complete input would fail because it would try to parse
|
||||
multiple records at once. Likewise, using "pretty printing" JSON
|
||||
(`JSON_PRETTY_PRINT`) is not allowed because each JSON text is limited to exactly
|
||||
one line. On the other hand, values containing newline characters (such as the
|
||||
`comment` property in the above example) do not cause issues because each newline
|
||||
within a JSON string will be represented by a `\n` instead.
|
||||
|
||||
One common alternative to NDJSON would be Comma-Separated Values (CSV).
|
||||
If you want to process CSV files, you may want to take a look at the related
|
||||
project [clue/reactphp-csv](https://github.com/clue/reactphp-csv) instead:
|
||||
|
||||
```
|
||||
name,age,comment
|
||||
Alice,30,"Yes, I like cheese"
|
||||
Bob,50,"Hello
|
||||
World!"
|
||||
```
|
||||
|
||||
CSV may look slightly simpler, but this simplicity comes at a price. CSV is
|
||||
limited to untyped, two-dimensional data, so there's no standard way of storing
|
||||
any nested structures or to differentiate a boolean value from a string or
|
||||
integer. Field names are sometimes used, sometimes they're not
|
||||
(application-dependant). Inconsistent handling for fields that contain
|
||||
separators such as `,` or spaces or line breaks (see the `comment` field above)
|
||||
introduce additional complexity and its text encoding is usually undefined,
|
||||
Unicode (or UTF-8) is unlikely to be supported and CSV files often use ISO
|
||||
8859-1 encoding or some variant (again application-dependant).
|
||||
|
||||
While NDJSON helps avoiding many of CSV's shortcomings, it is still a
|
||||
(relatively) young format while CSV files have been used in production systems
|
||||
for decades. This means that if you want to interface with an existing system,
|
||||
you may have to rely on the format that's already supported. If you're building
|
||||
a new system, using NDJSON is an excellent choice as it provides a flexible way
|
||||
to process individual records using a common text-based format that can include
|
||||
any kind of structured data.
|
||||
|
||||
## Usage
|
||||
|
||||
### Decoder
|
||||
|
||||
The `Decoder` (parser) class can be used to make sure you only get back
|
||||
complete, valid JSON elements when reading from a stream.
|
||||
It wraps a given
|
||||
[`ReadableStreamInterface`](https://github.com/reactphp/stream#readablestreaminterface)
|
||||
and exposes its data through the same interface, but emits the JSON elements
|
||||
as parsed values instead of just chunks of strings:
|
||||
|
||||
```
|
||||
{"name":"test","active":true}
|
||||
{"name":"hello w\u00f6rld","active":true}
|
||||
```
|
||||
|
||||
```php
|
||||
$stdin = new React\Stream\ReadableResourceStream(STDIN);
|
||||
|
||||
$ndjson = new Clue\React\NDJson\Decoder($stdin);
|
||||
|
||||
$ndjson->on('data', function ($data) {
|
||||
// $data is a parsed element from the JSON stream
|
||||
// line 1: $data = (object)array('name' => 'test', 'active' => true);
|
||||
// line 2: $data = (object)array('name' => 'hello wörld', 'active' => true);
|
||||
var_dump($data);
|
||||
});
|
||||
```
|
||||
|
||||
ReactPHP's streams emit chunks of data strings and make no assumption about their lengths.
|
||||
These chunks do not necessarily represent complete JSON elements, as an
|
||||
element may be broken up into multiple chunks.
|
||||
This class reassembles these elements by buffering incomplete ones.
|
||||
|
||||
The `Decoder` supports the same optional parameters as the underlying
|
||||
[`json_decode()`](https://www.php.net/manual/en/function.json-decode.php) function.
|
||||
This means that, by default, JSON objects will be emitted as a `stdClass`.
|
||||
This behavior can be controlled through the optional constructor parameters:
|
||||
|
||||
```php
|
||||
$ndjson = new Clue\React\NDJson\Decoder($stdin, true);
|
||||
|
||||
$ndjson->on('data', function ($data) {
|
||||
// JSON objects will be emitted as assoc arrays now
|
||||
});
|
||||
```
|
||||
|
||||
Additionally, the `Decoder` limits the maximum buffer size (maximum line
|
||||
length) to avoid buffer overflows due to malformed user input. Usually, there
|
||||
should be no need to change this value, unless you know you're dealing with some
|
||||
unreasonably long lines. It accepts an additional argument if you want to change
|
||||
this from the default of 64 KiB:
|
||||
|
||||
```php
|
||||
$ndjson = new Clue\React\NDJson\Decoder($stdin, false, 512, 0, 64 * 1024);
|
||||
```
|
||||
|
||||
If the underlying stream emits an `error` event or the plain stream contains
|
||||
any data that does not represent a valid NDJson stream,
|
||||
it will emit an `error` event and then `close` the input stream:
|
||||
|
||||
```php
|
||||
$ndjson->on('error', function (Exception $error) {
|
||||
// an error occured, stream will close next
|
||||
});
|
||||
```
|
||||
|
||||
If the underlying stream emits an `end` event, it will flush any incomplete
|
||||
data from the buffer, thus either possibly emitting a final `data` event
|
||||
followed by an `end` event on success or an `error` event for
|
||||
incomplete/invalid JSON data as above:
|
||||
|
||||
```php
|
||||
$ndjson->on('end', function () {
|
||||
// stream successfully ended, stream will close next
|
||||
});
|
||||
```
|
||||
|
||||
If either the underlying stream or the `Decoder` is closed, it will forward
|
||||
the `close` event:
|
||||
|
||||
```php
|
||||
$ndjson->on('close', function () {
|
||||
// stream closed
|
||||
// possibly after an "end" event or due to an "error" event
|
||||
});
|
||||
```
|
||||
|
||||
The `close(): void` method can be used to explicitly close the `Decoder` and
|
||||
its underlying stream:
|
||||
|
||||
```php
|
||||
$ndjson->close();
|
||||
```
|
||||
|
||||
The `pipe(WritableStreamInterface $dest, array $options = array(): WritableStreamInterface`
|
||||
method can be used to forward all data to the given destination stream.
|
||||
Please note that the `Decoder` emits decoded/parsed data events, while many
|
||||
(most?) writable streams expect only data chunks:
|
||||
|
||||
```php
|
||||
$ndjson->pipe($logger);
|
||||
```
|
||||
|
||||
For more details, see ReactPHP's
|
||||
[`ReadableStreamInterface`](https://github.com/reactphp/stream#readablestreaminterface).
|
||||
|
||||
### Encoder
|
||||
|
||||
The `Encoder` (serializer) class can be used to make sure anything you write to
|
||||
a stream ends up as valid JSON elements in the resulting NDJSON stream.
|
||||
It wraps a given
|
||||
[`WritableStreamInterface`](https://github.com/reactphp/stream#writablestreaminterface)
|
||||
and accepts its data through the same interface, but handles any data as complete
|
||||
JSON elements instead of just chunks of strings:
|
||||
|
||||
```php
|
||||
$stdout = new React\Stream\WritableResourceStream(STDOUT);
|
||||
|
||||
$ndjson = new Clue\React\NDJson\Encoder($stdout);
|
||||
|
||||
$ndjson->write(array('name' => 'test', 'active' => true));
|
||||
$ndjson->write(array('name' => 'hello wörld', 'active' => true));
|
||||
```
|
||||
```
|
||||
{"name":"test","active":true}
|
||||
{"name":"hello w\u00f6rld","active":true}
|
||||
```
|
||||
|
||||
The `Encoder` supports the same parameters as the underlying
|
||||
[`json_encode()`](https://www.php.net/manual/en/function.json-encode.php) function.
|
||||
This means that, by default, Unicode characters will be escaped in the output.
|
||||
This behavior can be controlled through the optional constructor parameters:
|
||||
|
||||
```php
|
||||
$ndjson = new Clue\React\NDJson\Encoder($stdout, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);
|
||||
|
||||
$ndjson->write('hello wörld');
|
||||
```
|
||||
```
|
||||
"hello wörld"
|
||||
```
|
||||
|
||||
Note that trying to pass the `JSON_PRETTY_PRINT` option will yield an
|
||||
`InvalidArgumentException` because it is not compatible with NDJSON.
|
||||
|
||||
If the underlying stream emits an `error` event or the given data contains
|
||||
any data that can not be represented as a valid NDJSON stream,
|
||||
it will emit an `error` event and then `close` the input stream:
|
||||
|
||||
```php
|
||||
$ndjson->on('error', function (Exception $error) {
|
||||
// an error occured, stream will close next
|
||||
});
|
||||
```
|
||||
|
||||
If either the underlying stream or the `Encoder` is closed, it will forward
|
||||
the `close` event:
|
||||
|
||||
```php
|
||||
$ndjson->on('close', function () {
|
||||
// stream closed
|
||||
// possibly after an "end" event or due to an "error" event
|
||||
});
|
||||
```
|
||||
|
||||
The `end(mixed $data = null): void` method can be used to optionally emit
|
||||
any final data and then soft-close the `Encoder` and its underlying stream:
|
||||
|
||||
```php
|
||||
$ndjson->end();
|
||||
```
|
||||
|
||||
The `close(): void` method can be used to explicitly close the `Encoder` and
|
||||
its underlying stream:
|
||||
|
||||
```php
|
||||
$ndjson->close();
|
||||
```
|
||||
|
||||
For more details, see ReactPHP's
|
||||
[`WritableStreamInterface`](https://github.com/reactphp/stream#writablestreaminterface).
|
||||
|
||||
## Install
|
||||
|
||||
The recommended way to install this library is [through Composer](https://getcomposer.org/).
|
||||
[New to Composer?](https://getcomposer.org/doc/00-intro.md)
|
||||
|
||||
This project follows [SemVer](https://semver.org/).
|
||||
This will install the latest supported version:
|
||||
|
||||
```bash
|
||||
composer require clue/ndjson-react:^1.3
|
||||
```
|
||||
|
||||
See also the [CHANGELOG](CHANGELOG.md) for details about version upgrades.
|
||||
|
||||
This project aims to run on any platform and thus does not require any PHP
|
||||
extensions and supports running on legacy PHP 5.3 through current PHP 8+ and
|
||||
HHVM.
|
||||
It's *highly recommended to use the latest supported PHP version* for this project.
|
||||
|
||||
## Tests
|
||||
|
||||
To run the test suite, you first need to clone this repo and then install all
|
||||
dependencies [through Composer](https://getcomposer.org/):
|
||||
|
||||
```bash
|
||||
composer install
|
||||
```
|
||||
|
||||
To run the test suite, go to the project root and run:
|
||||
|
||||
```bash
|
||||
vendor/bin/phpunit
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
This project is released under the permissive [MIT license](LICENSE).
|
||||
|
||||
> Did you know that I offer custom development services and issuing invoices for
|
||||
sponsorships of releases and for contributions? Contact me (@clue) for details.
|
||||
|
||||
## More
|
||||
|
||||
* If you want to learn more about processing streams of data, refer to the documentation of
|
||||
the underlying [react/stream](https://github.com/reactphp/stream) component.
|
||||
|
||||
* If you want to process compressed NDJSON files (`.ndjson.gz` file extension),
|
||||
you may want to use [clue/reactphp-zlib](https://github.com/clue/reactphp-zlib)
|
||||
on the compressed input stream before passing the decompressed stream to the NDJSON decoder.
|
||||
|
||||
* If you want to create compressed NDJSON files (`.ndjson.gz` file extension),
|
||||
you may want to use [clue/reactphp-zlib](https://github.com/clue/reactphp-zlib)
|
||||
on the resulting NDJSON encoder output stream before passing the compressed
|
||||
stream to the file output stream.
|
||||
|
||||
* If you want to concurrently process the records from your NDJSON stream,
|
||||
you may want to use [clue/reactphp-flux](https://github.com/clue/reactphp-flux)
|
||||
to concurrently process many (but not too many) records at once.
|
||||
|
||||
* If you want to process structured data in the more common text-based format,
|
||||
you may want to use [clue/reactphp-csv](https://github.com/clue/reactphp-csv)
|
||||
to process Comma-Separated-Values (CSV) files (`.csv` file extension).
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"name": "clue/ndjson-react",
|
||||
"description": "Streaming newline-delimited JSON (NDJSON) parser and encoder for ReactPHP.",
|
||||
"keywords": ["NDJSON", "newline", "JSON", "jsonlines", "streaming", "ReactPHP"],
|
||||
"homepage": "https://github.com/clue/reactphp-ndjson",
|
||||
"license": "MIT",
|
||||
"authors": [
|
||||
{
|
||||
"name": "Christian Lück",
|
||||
"email": "christian@clue.engineering"
|
||||
}
|
||||
],
|
||||
"require": {
|
||||
"php": ">=5.3",
|
||||
"react/stream": "^1.2"
|
||||
},
|
||||
"require-dev": {
|
||||
"phpunit/phpunit": "^9.5 || ^5.7 || ^4.8.35",
|
||||
"react/event-loop": "^1.2"
|
||||
},
|
||||
"autoload": {
|
||||
"psr-4": {
|
||||
"Clue\\React\\NDJson\\": "src/"
|
||||
}
|
||||
},
|
||||
"autoload-dev": {
|
||||
"psr-4": {
|
||||
"Clue\\Tests\\React\\NDJson\\": "tests/"
|
||||
}
|
||||
}
|
||||
}
|
||||
+166
@@ -0,0 +1,166 @@
|
||||
<?php
|
||||
|
||||
namespace Clue\React\NDJson;
|
||||
|
||||
use Evenement\EventEmitter;
|
||||
use React\Stream\ReadableStreamInterface;
|
||||
use React\Stream\Util;
|
||||
use React\Stream\WritableStreamInterface;
|
||||
|
||||
/**
|
||||
* The Decoder / Parser reads from a plain stream and emits data objects for each JSON element
|
||||
*/
|
||||
class Decoder extends EventEmitter implements ReadableStreamInterface
|
||||
{
|
||||
private $input;
|
||||
private $assoc;
|
||||
private $depth;
|
||||
private $options;
|
||||
/** @var int */
|
||||
private $maxlength;
|
||||
|
||||
private $buffer = '';
|
||||
private $closed = false;
|
||||
|
||||
/**
|
||||
* @param ReadableStreamInterface $input
|
||||
* @param bool $assoc
|
||||
* @param int $depth
|
||||
* @param int $options (requires PHP 5.4+)
|
||||
* @param int $maxlength
|
||||
* @throws \BadMethodCallException
|
||||
*/
|
||||
public function __construct(ReadableStreamInterface $input, $assoc = false, $depth = 512, $options = 0, $maxlength = 65536)
|
||||
{
|
||||
// @codeCoverageIgnoreStart
|
||||
if ($options !== 0 && \PHP_VERSION < 5.4) {
|
||||
throw new \BadMethodCallException('Options parameter is only supported on PHP 5.4+');
|
||||
}
|
||||
if (\defined('JSON_THROW_ON_ERROR')) {
|
||||
$options = $options & ~\JSON_THROW_ON_ERROR;
|
||||
}
|
||||
// @codeCoverageIgnoreEnd
|
||||
|
||||
$this->input = $input;
|
||||
|
||||
if (!$input->isReadable()) {
|
||||
$this->close();
|
||||
return;
|
||||
}
|
||||
|
||||
$this->assoc = $assoc;
|
||||
$this->depth = $depth;
|
||||
$this->options = $options;
|
||||
$this->maxlength = $maxlength;
|
||||
|
||||
$this->input->on('data', array($this, 'handleData'));
|
||||
$this->input->on('end', array($this, 'handleEnd'));
|
||||
$this->input->on('error', array($this, 'handleError'));
|
||||
$this->input->on('close', array($this, 'close'));
|
||||
}
|
||||
|
||||
public function isReadable()
|
||||
{
|
||||
return !$this->closed;
|
||||
}
|
||||
|
||||
public function close()
|
||||
{
|
||||
if ($this->closed) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->closed = true;
|
||||
$this->buffer = '';
|
||||
|
||||
$this->input->close();
|
||||
|
||||
$this->emit('close');
|
||||
$this->removeAllListeners();
|
||||
}
|
||||
|
||||
public function pause()
|
||||
{
|
||||
$this->input->pause();
|
||||
}
|
||||
|
||||
public function resume()
|
||||
{
|
||||
$this->input->resume();
|
||||
}
|
||||
|
||||
public function pipe(WritableStreamInterface $dest, array $options = array())
|
||||
{
|
||||
Util::pipe($this, $dest, $options);
|
||||
|
||||
return $dest;
|
||||
}
|
||||
|
||||
/** @internal */
|
||||
public function handleData($data)
|
||||
{
|
||||
if (!\is_string($data)) {
|
||||
$this->handleError(new \UnexpectedValueException('Expected stream to emit string, but got ' . \gettype($data)));
|
||||
return;
|
||||
}
|
||||
|
||||
$this->buffer .= $data;
|
||||
|
||||
// keep parsing while a newline has been found
|
||||
while (($newline = \strpos($this->buffer, "\n")) !== false && $newline <= $this->maxlength) {
|
||||
// read data up until newline and remove from buffer
|
||||
$data = (string)\substr($this->buffer, 0, $newline);
|
||||
$this->buffer = (string)\substr($this->buffer, $newline + 1);
|
||||
|
||||
// decode data with options given in ctor
|
||||
// @codeCoverageIgnoreStart
|
||||
if ($this->options === 0) {
|
||||
$data = \json_decode($data, $this->assoc, $this->depth);
|
||||
} else {
|
||||
assert(\PHP_VERSION_ID >= 50400);
|
||||
$data = \json_decode($data, $this->assoc, $this->depth, $this->options);
|
||||
}
|
||||
// @codeCoverageIgnoreEnd
|
||||
|
||||
// abort stream if decoding failed
|
||||
if ($data === null && \json_last_error() !== \JSON_ERROR_NONE) {
|
||||
// @codeCoverageIgnoreStart
|
||||
if (\PHP_VERSION_ID > 50500) {
|
||||
$errstr = \json_last_error_msg();
|
||||
} elseif (\json_last_error() === \JSON_ERROR_SYNTAX) {
|
||||
$errstr = 'Syntax error';
|
||||
} else {
|
||||
$errstr = 'Unknown error';
|
||||
}
|
||||
// @codeCoverageIgnoreEnd
|
||||
return $this->handleError(new \RuntimeException('Unable to decode JSON: ' . $errstr, \json_last_error()));
|
||||
}
|
||||
|
||||
$this->emit('data', array($data));
|
||||
}
|
||||
|
||||
if (isset($this->buffer[$this->maxlength])) {
|
||||
$this->handleError(new \OverflowException('Buffer size exceeded'));
|
||||
}
|
||||
}
|
||||
|
||||
/** @internal */
|
||||
public function handleEnd()
|
||||
{
|
||||
if ($this->buffer !== '') {
|
||||
$this->handleData("\n");
|
||||
}
|
||||
|
||||
if (!$this->closed) {
|
||||
$this->emit('end');
|
||||
$this->close();
|
||||
}
|
||||
}
|
||||
|
||||
/** @internal */
|
||||
public function handleError(\Exception $error)
|
||||
{
|
||||
$this->emit('error', array($error));
|
||||
$this->close();
|
||||
}
|
||||
}
|
||||
+144
@@ -0,0 +1,144 @@
|
||||
<?php
|
||||
|
||||
namespace Clue\React\NDJson;
|
||||
|
||||
use Evenement\EventEmitter;
|
||||
use React\Stream\WritableStreamInterface;
|
||||
|
||||
/**
|
||||
* The Encoder / Serializer can be used to write any value, encode it as a JSON text and forward it to an output stream
|
||||
*/
|
||||
class Encoder extends EventEmitter implements WritableStreamInterface
|
||||
{
|
||||
private $output;
|
||||
private $options;
|
||||
private $depth;
|
||||
|
||||
private $closed = false;
|
||||
|
||||
/**
|
||||
* @param WritableStreamInterface $output
|
||||
* @param int $options
|
||||
* @param int $depth (requires PHP 5.5+)
|
||||
* @throws \InvalidArgumentException
|
||||
* @throws \BadMethodCallException
|
||||
*/
|
||||
public function __construct(WritableStreamInterface $output, $options = 0, $depth = 512)
|
||||
{
|
||||
// @codeCoverageIgnoreStart
|
||||
if (\defined('JSON_PRETTY_PRINT') && $options & \JSON_PRETTY_PRINT) {
|
||||
throw new \InvalidArgumentException('Pretty printing not available for NDJSON');
|
||||
}
|
||||
if ($depth !== 512 && \PHP_VERSION < 5.5) {
|
||||
throw new \BadMethodCallException('Depth parameter is only supported on PHP 5.5+');
|
||||
}
|
||||
if (\defined('JSON_THROW_ON_ERROR')) {
|
||||
$options = $options & ~\JSON_THROW_ON_ERROR;
|
||||
}
|
||||
// @codeCoverageIgnoreEnd
|
||||
|
||||
$this->output = $output;
|
||||
|
||||
if (!$output->isWritable()) {
|
||||
$this->close();
|
||||
return;
|
||||
}
|
||||
|
||||
$this->options = $options;
|
||||
$this->depth = $depth;
|
||||
|
||||
$this->output->on('drain', array($this, 'handleDrain'));
|
||||
$this->output->on('error', array($this, 'handleError'));
|
||||
$this->output->on('close', array($this, 'close'));
|
||||
}
|
||||
|
||||
public function write($data)
|
||||
{
|
||||
if ($this->closed) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// we have to handle PHP warnings for legacy PHP < 5.5
|
||||
// certain values (such as INF etc.) emit a warning, but still encode successfully
|
||||
// @codeCoverageIgnoreStart
|
||||
if (\PHP_VERSION_ID < 50500) {
|
||||
$errstr = null;
|
||||
\set_error_handler(function ($_, $error) use (&$errstr) {
|
||||
$errstr = $error;
|
||||
});
|
||||
|
||||
// encode data with options given in ctor (depth not supported)
|
||||
$data = \json_encode($data, $this->options);
|
||||
|
||||
// always check error code and match missing error messages
|
||||
\restore_error_handler();
|
||||
$errno = \json_last_error();
|
||||
if (\defined('JSON_ERROR_UTF8') && $errno === \JSON_ERROR_UTF8) {
|
||||
// const JSON_ERROR_UTF8 added in PHP 5.3.3, but no error message assigned in legacy PHP < 5.5
|
||||
// this overrides PHP 5.3.14 only: https://3v4l.org/IGP8Z#v5314
|
||||
$errstr = 'Malformed UTF-8 characters, possibly incorrectly encoded';
|
||||
} elseif ($errno !== \JSON_ERROR_NONE && $errstr === null) {
|
||||
// error number present, but no error message applicable
|
||||
$errstr = 'Unknown error';
|
||||
}
|
||||
|
||||
// abort stream if encoding fails
|
||||
if ($errno !== \JSON_ERROR_NONE || $errstr !== null) {
|
||||
$this->handleError(new \RuntimeException('Unable to encode JSON: ' . $errstr, $errno));
|
||||
return false;
|
||||
}
|
||||
} else {
|
||||
// encode data with options given in ctor
|
||||
$data = \json_encode($data, $this->options, $this->depth);
|
||||
|
||||
// abort stream if encoding fails
|
||||
if ($data === false && \json_last_error() !== \JSON_ERROR_NONE) {
|
||||
$this->handleError(new \RuntimeException('Unable to encode JSON: ' . \json_last_error_msg(), \json_last_error()));
|
||||
return false;
|
||||
}
|
||||
}
|
||||
// @codeCoverageIgnoreEnd
|
||||
|
||||
return $this->output->write($data . "\n");
|
||||
}
|
||||
|
||||
public function end($data = null)
|
||||
{
|
||||
if ($data !== null) {
|
||||
$this->write($data);
|
||||
}
|
||||
|
||||
$this->output->end();
|
||||
}
|
||||
|
||||
public function isWritable()
|
||||
{
|
||||
return !$this->closed;
|
||||
}
|
||||
|
||||
public function close()
|
||||
{
|
||||
if ($this->closed) {
|
||||
return;
|
||||
}
|
||||
|
||||
$this->closed = true;
|
||||
$this->output->close();
|
||||
|
||||
$this->emit('close');
|
||||
$this->removeAllListeners();
|
||||
}
|
||||
|
||||
/** @internal */
|
||||
public function handleDrain()
|
||||
{
|
||||
$this->emit('drain');
|
||||
}
|
||||
|
||||
/** @internal */
|
||||
public function handleError(\Exception $error)
|
||||
{
|
||||
$this->emit('error', array($error));
|
||||
$this->close();
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user