buffered improvements for the reader #1

New issue

Open

opened 2020-04-02 00:14:47 +00:00 by howl · 0 comments

howl commented

2020-04-02 00:14:47 +00:00

(Migrated from zxq.co)

Mostly a note to self for next time like I feel like working on this.

Tonight I just modified the WriteChain to write in buffers instead of at every operation. This yielded great performance improvements. Of course, we could do this on Reader as well, except that it wouldn't be backwards compatible, seeing as we don't know beforehand how much data we will need to read.

encoding/binary has an advantage over us, because it knows beforehand how many values it needs to read (it has one function with one array). We, of course, can't do that: the only way to know the amount of stuff to read beforehand is by having a "ReadChain" which holds references to the value to which to decode. And then a whole other bunch of problems come up.

(Ironically, I remember when I started working on binary I intended to have ReadChain, but then discarded the idea)

The encoding/json solution is to buffer the data, but let users have https://golang.org/pkg/encoding/json/#Decoder.Buffered to read from the already-buffered data. Which could be adequate.

A non breaking way could be to have a parameter like Length in the reader, which basically tells it how much there is still to read. (And could be -1 to always read max buffer). Problem with this, though, is that any if we add costs time, so if it comes to it we'll just bump a major version.

Mostly a note to self for next time like I feel like working on this. Tonight I just modified the WriteChain to write in buffers instead of at every operation. This yielded great performance improvements. Of course, we could do this on Reader as well, except that it wouldn't be backwards compatible, seeing as we don't know beforehand how much data we will need to read. encoding/binary has an advantage over us, because it knows beforehand how many values it needs to read (it has one function with one array). We, of course, can't do that: the only way to know the amount of stuff to read beforehand is by having a "ReadChain" which holds references to the value to which to decode. And then a whole other bunch of problems come up. (Ironically, I remember when I started working on binary I intended to have ReadChain, but then discarded the idea) The encoding/json solution is to buffer the data, but let users have https://golang.org/pkg/encoding/json/#Decoder.Buffered to read from the already-buffered data. Which could be adequate. A non breaking way could be to have a parameter like `Length` in the reader, which basically tells it how much there is still to read. (And could be -1 to always read max buffer). Problem with this, though, is that any `if` we add costs time, so if it comes to it we'll just bump a major version.

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

howl/binary#1

No description provided.

Rows
Columns