public-offering
/
pad.pub0.org
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

ImportEtherpad: Batch database writes

pull/5513/head
Richard Hansen 2022-04-19 16:47:54 -04:00
parent c1652fd695
commit 44fd70491d
2 changed files with 7 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
CHANGELOG.md
View File

@ -39,6 +39,8 @@
timeouts with large pads. timeouts with large pads.
* Exporting a large pad to `.etherpad` format should be faster thanks to bulk * Exporting a large pad to `.etherpad` format should be faster thanks to bulk
database record fetches. database record fetches.
* When importing an `.etherpad` file, records are now saved to the database in
batches to avoid database timeouts with large pads.
#### For plugin authors #### For plugin authors

9
src/node/utils/ImportEtherpad.js
View File

@ -105,8 +105,9 @@ exports.setPadRaw = async (padId, r, authorId = '') => {
await padDb.close(); await padDb.close();
} }
await Promise.all([ const writeOps = (function* () {
...[...data].map(([k, v]) => q.pushAsync(() => db.set(k, v))), for (const [k, v] of data) yield db.set(k, v);
...[...existingAuthors].map((a) => q.pushAsync(() => authorManager.addPad(a, padId))), for (const a of existingAuthors) yield authorManager.addPad(a, padId);
]); })();
for (const op of new Stream(writeOps).batch(100).buffer(99)) await op;
}; };