By specification [0], the if-modified-since HTTP header sent by browsers does
not include milliseconds.
Before this patch, let's say a file was generate at time:
t_real-file = 2020-03-22T02:15:53.548Z (note the fractional seconds)
When issuing a conditional request, the browser would truncate the fractional
part, and only request an if-modified-since with this contents:
t_if-modified-since = 2020-03-22T02:15:53.000Z
The minify() function would return HTTP/304 only if
t_if-modified-since >= t_real-file, but this would never be true unless, by
chance, a file was generated at XX.000Z.
This resulted in that file being minified/compressed again and resent to the
client for no reason. After this patch, the server correctly responds with
HTTP/304 without doing any computation, and the browser uses the cached file.
[0] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since
No variation in # of security vulnerabilities.
The output of `npm audit` goes from this:
found 7 vulnerabilities (5 low, 2 high) in 13881 scanned packages
7 vulnerabilities require manual review. See the full report for details.
To this:
found 7 vulnerabilities (5 low, 2 high) in 13795 scanned packages
7 vulnerabilities require manual review. See the full report for details.
This is a dev dependency. No impact in production.
After this upgrade the backend tests ("cd src; npm test") still pass.
This fixes 2 reported security vulnerabilities.
The output of `npm audit` goes from this:
found 9 vulnerabilities (7 low, 2 high) in 13707 scanned packages
run `npm audit fix` to fix 1 of them.
1 vulnerability requires semver-major dependency updates.
7 vulnerabilities require manual review. See the full report for details.
To this:
found 7 vulnerabilities (5 low, 2 high) in 13881 scanned packages
7 vulnerabilities require manual review. See the full report for details.
This is a dev dependency. No impact in production.
After this upgrade the backend tests ("cd src; npm test") still pass.
This fixes 1 reported security vulnerability.
The output of `npm audit` goes from this:
found 10 vulnerabilities (8 low, 2 high) in 13390 scanned packages
run `npm audit fix` to fix 2 of them.
1 vulnerability requires semver-major dependency updates.
7 vulnerabilities require manual review. See the full report for details.
To this:
found 9 vulnerabilities (7 low, 2 high) in 13707 scanned packages
run `npm audit fix` to fix 1 of them.
1 vulnerability requires semver-major dependency updates.
7 vulnerabilities require manual review. See the full report for details.
This fixes 107 reported security vulnerabilites.
The output of `npm audit` goes from this:
found 117 vulnerabilities (115 low, 2 high) in 13365 scanned packages
run `npm audit fix` to fix 6 of them.
1 vulnerability requires semver-major dependency updates.
110 vulnerabilities require manual review. See the full report for details.
To this:
found 10 vulnerabilities (8 low, 2 high) in 13390 scanned packages
run `npm audit fix` to fix 2 of them.
1 vulnerability requires semver-major dependency updates.
7 vulnerabilities require manual review. See the full report for details.
The changelog for npm can be read at https://github.com/npm/cli/releases
CleanCSS 3.4.19 had a Regex Denial of Service vulnerability and has to be
updated. The major version bump requires the following changes:
1. Disabling rebase is necessary because otherwise the URLs for the web fonts
become wrong;
EXAMPLE 1:
/static/css/src/static/font/fontawesome-etherpad.woff
instead of
/static/font/fontawesome-etherpad.woff
EXAMPLE 2 (this is more surprising):
/p/src/static/font/opendyslexic.otf
instead of
/static/font/opendyslexic.otf
2. CleanCSS.minify() can either receive a string containing the CSS, or an array
of strings. In that case each array element is interpreted as an absolute
local path from which the CSS file is read.
In version 4.x, CleanCSS API was simplified, eliminating the relativeTo
parameter, and thus we cannot use our already loaded "content" argument, but
we have to wrap the absolute path to the CSS in an array and ask the library
to read it by itself.
Fixes#3616.
The next commit will update a dependency in package.json. This commit only
refreshes package-lock.json recalculating it at the current date.
In this way we will be sure that any modifications in package-lock.json in the
next commit will be only due to the package.json change.
Should this commit introduce any regression, we would know for certain that one
of our dependencies is not correctly honoring their semver promises.
For some reason authorInfo is sometimes null, and therefore it is not possible
to get colorId from it.
This resulted in the following stack trace:
[2020-03-16 09:27:17.291] [ERROR] console - (node:1746) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'colorId' of null
at <BASEDIR>/src/node/handler/PadMessageHandler.js:1199:37
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async Promise.all (index 0)
at async handleClientReady (<BASEDIR>/src/node/handler/PadMessageHandler.js:1171:5)
[2020-03-16 09:27:17.291] [ERROR] console - (node:1746) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 76)
[2020-03-16 09:27:19.034] [WARN] message - Dropped message, USERINFO_UPDATE Session not ready.[object Object]
Which is due to a bug in Etherpad that we are not going to solve now.
As a workaround, when this happens, let's set the username to "Anonymous" (if
it is not already set), and colorId to the fixed value "#daf0b2". Warning
messages are written in the logs to signal this condition.
This is no definitive solution, but fixes#3612 (via a workaround).
Before this patch, visiting the read-only URL for a random pad would remove
the "Save Revision" (the "star" icon) from all the other RW pads. The only way
to make it appear again was to restart the server.
This change does not fix the underlying bug: after visiting a read only link
the "star" button would still disapper, but it is explictly reinserted via an
ad-hoc condition.
Fixes#3702
That commit (merged recently with PR #3622) was part of an effort to fix#3620,
but introduced a very bad bug that broke the cursor behaviour when pressing
space, making the program unusable.
This commit completes the revert of PR #3622 and fixes#3728.
--HG--
branch : revert-3622
By specification, when settings.allowUnknownFileEnds is true and the user tries
to import a file with an unknown extension (this includes no extension),
Etherpad tries to import it as txt.
This broke in Etherpad 1.8.0, that abruptly terminates the processing with an
UnhandledPromiseRejectionWarning.
This patch restores the intended behaviour, and allows to import as text a file
with an unknown extension (on no extension).
In order to catch the UnhandledPromiseRejectionWarning we had to use
fsp_rename(), which is declared earlier in the code and is promised based
instead of fs.rename(), which is callback based.
Fixes#3710.
Before this change, invoking a non existing API method would return an HTTP/200
response with a JSON payload {"code":3,"message":"no such function"}.
This commit changes the HTTP status code to 404, leaving the payload as-is.
Before:
curl --verbose "http://localhost:9001/api/1/notExisting?apikey=ABCDEF"
< HTTP/1.1 200 OK
< X-Powered-By: Express
[...]
{"code":3,"message":"no such function","data":null}
After:
curl --verbose "http://localhost:9001/api/1/notExisting?apikey=ABCDEF"
< HTTP/1.1 404 OK
< X-Powered-By: Express
[...]
{"code":3,"message":"no such function","data":null}
Fixes#3546.
Before this change there was always a single toolbar on the top, with both
Colibris and the legacy skin. When the screen size was reduced:
- the legacy skin would compact the icons in the toolbar (this was fine,
indeed);
- Colibris would hide some formatting icons. This would hamper the functionality
for mobile users.
After this change both the skins work in the same way, which is the following:
- when the screen gets smaller the right toolbar (the one with "export",
"timeslider", and other buttons) goes to the bottom of the screen;
- when there are many icons, the toolbar keeps all of them, and to see them the
user must drag the toolbar.
This behaviour will probably be changed before release, opting instead to show
a "+" button when there is an overflow, since this appears to be more
discoverable (see the discusison in #3697).
Do not tested with custom toolbar elements (toolbar.left and toolbar.right
configuration items in settings.json).
Fixes#3697.
Steps to reproduce (via HTTP API):
1. create a group via createGroup()
2. create a group pad inside that group via createGroupPad()
3. make that pad public calling setPublicStatus(true)
4. access the pad via a clean web browser (with no sessions)
5. UnhandledPromiseRejectionWarning: apierror: sessionID does not exist
This was due to an overlook in 769933786cea: "apierror: sessionID does not
exist" may be a legal condition if we are also visiting a public pad. The
function that could throw that error was sessionManager.getSessionInfo(), and
thus it needed to be inside the try...catch block.
Please note that calling getText() on the pad always return the pad contents,
*even for non-public pads*, because the API bypasses the security checks and
directly talks to the DB layer.
Fixes#3600.
Note by muxator:
This commit introduced a copied & modified version of the testing files
loadSettings.js and pad.js.
It's Christmas night, and we want to shipt this feature, so I merged it anyway,
adding a note in both the original and copied files so that hopefully someone
in the distant future is going to merge them back again.
This fixes some security vulnerabilites, among them an arbitrary file overwrite.
The output of `npm audit` goes from this:
found 17 vulnerabilities (15 low, 2 high) in 13344 scanned packages
run `npm audit fix` to fix 6 of them.
1 vulnerability requires semver-major dependency updates.
10 vulnerabilities require manual review. See the full report for details.
To this:
found 5 vulnerabilities (3 low, 2 high) in 13370 scanned packages
1 vulnerability requires semver-major dependency updates.
4 vulnerabilities require manual review. See the full report for details.
Changelog:
- https://github.com/npm/cli/releases
6.13.4 (2019-12-11)
BUGFIXES
320ac9aee npm/bin-links#12 npm/gentle-fs#7 Do not remove global bin/man links inappropriately (@isaacs)
DEPENDENCIES
52fd21061 gentle-fs@2.3.0 (@isaacs)
d06f5c0b0 bin-links@1.1.6 (@isaacs)
6.13.3 (2019-12-09)
DEPENDENCIES
19ce061a2 bin-links@1.1.5 Properly normalize, sanitize, and verify bin entries in package.json.
59c836aae npm-packlist@1.4.7
fb4ecd7d2 pacote@9.5.11
5f33040 #476npm/pacote#22npm/pacote#14 fix: Do not drop perms in git when not root (isaacs, @darcyclarke)
6f229f7 sanitize and normalize package bin field (isaacs)
1743cb339 read-package-json@2.1.1
6.13.2 (2019-12-03)
BUG FIXES
4429645b3 #546 fix docs target typo (@richardlau)
867642942 #142 fix(packageRelativePath): fix 'where' for file deps (@larsgw)
d480f2c17 #527 Revert "windows: Add preliminary WSL support for npm and npx" (@craigloewen-msft)
e4b97962e #504 remove unnecessary package.json read when reading shrinkwrap (@Lighting-Jack)
1c65d26ac #501 fix(fund): open url for string shorthand (@ruyadorno)
ae7afe565 #263 Don't log error message if git tagging is disabled (@woppa684)
4c1b16f6a #182 Warn the user that it is uninstalling npm-install (@Hoidberg)
The mechanism used for determining if the application is being served over SSL
is wrapped by the "express-session" library for "express_sid", and manual for
the "language" cookie, but it's very similar in both cases.
The "secure" flag is set if one of these is true:
1. we are directly serving Etherpad over SSL using the native nodejs
functionality, via the "ssl" options in settings.json
2. Etherpad is being served in plaintext by nodejs, but we are using a reverse
proxy for terminating the SSL for us;
In this case, the user has to be instructed to properly set trustProxy: true
in settings.json, and the information wheter the application is over SSL or
not will be extracted from the X-Forwarded-Proto HTTP header.
Please note that this will not be compatible with applications being served over
http and https at the same time.
The change on webaccess.js amends 009b61b338, which did not work when the SSL
termination was performed by a reverse proxy.
Reference for automatic "express_sid" configuration:
https://github.com/expressjs/session/blob/v1.17.0/README.md#cookiesecureCloses#3561.
The "io" cookie is created by socket.io, and its purpose is to offer an handle
to perform load balancing with session stickiness when the library falls back to
long polling or below.
In Etherpad's case, if an operator needs to load balance, he can use the
"express_sid" cookie, and thus "io" is of no use.
Moreover, socket.io API does not offer a way of setting the "secure" flag on it,
and thus is a liability.
Let's simply nuke it.
References:
https://socket.io/docs/using-multiple-nodes/#Sticky-load-balancinghttps://github.com/socketio/socket.io/issues/2276#issuecomment-147184662 (not totally true, actually, see above)
Revision 5879037ddc fixed a security bug, but introduced a regression, where
on page load the js console showed:
ReferenceError: require is not defined
The reason was that the fix called require('../static/js/pad_utils') to load a
module at a time when require() was still not defined.
This change anticipates the loading of require-kernel, and manually loads
pad_utils.
The fix proposed in #3670 by aaron-costello, which seemed to do the right
thing, anticipating the configuration phase of require-kernel, did not work.
It had to be declined and replaced by this (less elegant) change.
This upgrade solves the high-severity vulnerabilities regarding
https-proxy-agent that were still present in 8e6bca456f.
The output of `npm audit` goes from this:
found 29 vulnerabilities (3 low, 26 high) in 13338 scanned packages
run `npm audit fix` to fix 4 of them.
1 vulnerability requires semver-major dependency updates.
24 vulnerabilities require manual review. See the full report for details.
To this:
found 5 vulnerabilities (3 low, 2 high) in 13338 scanned packages
1 vulnerability requires semver-major dependency updates.
4 vulnerabilities require manual review. See the full report for details.
Changelog:
- https://github.com/npm/cli/releases
6.13.1 (2019-11-18)
BUG FIXES
938d6124d #472 fix(fund): support funding string shorthand (@ruyadorno)
b49c5535b #471 should not publish tap-snapshot folder (@ruyadorno)
3471d5200 #253 Add preliminary WSL support for npm and npx (@infinnie)
3ef295f23 #486 print quick audit report for human output (@isaacs)
TESTING
dbbf977ac #278 added workflow to trigger and run benchmarks (@mikemimik)
b4f5e3825 #457 feat(docs): adding tests and updating docs to reflect changes in registry teams API. (@nomadtechie)
454c7dd60 #456 fix git configs for git 2.23 and above (@isaacs)
DEPENDENCIES
661d86cd2 make-fetch-happen@5.0.2 (@claudiahdz)
6.13.0 (2019-11-05)
NEW FEATURES
4414b06d9 #273 add fund command (@ruyadorno)
BUG FIXES
e4455409f #281 delete ps1 files on package removal (@NoDocCat)
cd14d4701 #279 update supported node list to remove v6.0, v6.1, v9.0 - v9.2 (@ljharb)
DEPENDENCIES
a37296b20 pacote@9.5.9
d3cb3abe8 read-cmd-shim@1.0.5
TESTING
688cd97be #272 use github actions for CI (@JasonEtco)
9a2d8af84 #240 Clean up some flakiness and inconsistency (@isaacs)
Added `rel="noreferrer"` to automatically generated links in the main pad window
as well as the chat window.
`rel="noreferrer"` is part of the HTML5 standard. While browser support isn't
100%, it's better than nothing. Future alternative solutions with wider browser
support, such as intermediary redirect pages, are unaffected by this change.
https://html.spec.whatwg.org/multipage/links.html#link-type-noreferrer
This commit was originally part of https://github.com/ether/etherpad-lite/pull/2498
The change that implemented #3648 (7c099fef5e) was incorrect, and resulted
in disabling every user at startup.
The problem was twofold:
1. _.filter() on an object returns an array of the object's enumerable values
and strips out the keys, see: https://stackoverflow.com/questions/11697702/how-to-use-underscore-js-filter-with-an-object
To filter an object, the function that needs to be used is _.pick();
2. The logic condition on userProperties.password was plain wrong (it should
have been an AND instead of an OR).
This change corrects 1) and 2), and writes more specific logs when something
goes wrong.
Closes#3661.
This change reverts c4918efc1b, and basically negates what was done for #3396,
but aligns better with current practices in the nodejs ecosystem.
Pragmatically speaking, this will allow users, if they want, to use
npm-force-resolutions (https://github.com/rogeriochaves/npm-force-resolutions)
to manually fix security vulnerabilities.
We had a problem for that (see #3598), and - given the fragmented nature of
the nodejs ecosystem - it is reasonable to expect more issues like that one,
so it's better to be prepared.
Closes#3659.
The previous attempt to directly release 1.8.0 had to be hold back, and indeed
1.8.0 was never tagged.
Since 1.8.0 contains many changes, let's do a prerelease instead.
Closes#3660
Without this, on nodejs 10 and 12 (and maybe 8, not tested), Etherpad failed to
start, throwing the following error:
[2019-10-22 19:01:01.439] [ERROR] console - exception thrown: Maximum call stack size exceeded
[2019-10-22 19:01:01.439] [INFO] console - RangeError: Maximum call stack size exceeded
at Function.[Symbol.hasInstance] (<anonymous>)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:194:14)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
Fixes#3654.
This upgrade should be backward compatible, but still suffers form major
vulnerabilities in its https-proxy-agent transitive dependency (see
https://www.npmjs.com/advisories/1184).
Changelog:
- https://github.com/npm/cli/releases
6.12.0 (2019-10-08):
Now npm ci runs prepare scripts for git dependencies, and respects the
--no-optional argument. Warnings for engine mismatches are printed again.
Various other fixes and cleanups.
BUG FIXES
890b245dc #252 ci: add dirPacker to options (@claudiahdz)
f3299acd0 #257 npm.community#4792 warn message on engine mismatch
(@ruyadorno)
bbc92fb8f #259 npm.community#10288 Fix figgyPudding error in npm token
(@benblank)
70f54dcb5 #241 doctor: Make OK more consistent (@gemal)
FEATURES
ed993a29c #249 Add CI environment variables to user-agent (@isaacs)
f6b0459a4 #248 Add option to save package-lock without formatting Adds a new
config --format-package-lock, which defaults to true.
(@bl00mber)
DEPENDENCIES
0ca063c5d npm-lifecycle@3.1.4:
fix: filter functions and undefined out of makeEnv (@isaacs)
5df6b0ea2 libcipm@4.0.4:
fix: pack git directories properly (@claudiahdz)
respect no-optional argument (@cruzdanilo)
7e04f728c tar@4.4.12
5c380e5a3 stringify-package@1.0.1 (@isaacs)
62f2ca692 node-gyp@5.0.5 (@isaacs)
0ff0ea47a npm-install-checks@3.0.2 (@isaacs)
f46edae94 hosted-git-info@2.8.5 (@isaacs)
TESTING
44a2b036b #262 fix root-ownership race conditions in meta-test (@isaacs)
6.11.3 (2019-09-03):
Fix npm ci regressions and npm outdated depth.
BUG FIXES
235ed1d28 #239 Don't override user specified depth in outdated. Restores
ability to update packages using --depth as suggested by npm audit. (@G-Rath)
1fafb5151 #242 npm.community#9586 Revert "install: do not descend into
directory deps' child modules" (@isaacs)
cebf542e6 #243 npm.community#9720 ci: pass appropriate configs for file/dir
modes (@isaacs)
DEPENDENCIES
e5fbb7ed1 read-cmd-shim@1.0.4 (@claudiahdz)
23ce65616 npm-pick-manifest@3.0.2 (@claudiahdz)
6.11.2 (2019-08-22):
Fix a recent Windows regression, and two long-standing Windows bugs. Also,
get CI running on Windows, so these things are less likely in the future.
DEPENDENCIES
9778a1b87 cmd-shim@3.0.3: Fix regression where shims fail to preserve exit
code (@isaacs)
bf93e91d8 npm-package-arg@6.1.1: Properly handle git+file: urls on Windows
when a drive letter is included. (@isaacs)
BUGFIXES
6cc4cc66f escape args properly on Windows Bash Despite being bash, Node.js
running on windows git mingw bash still executes child processes
using cmd.exe. As a result, arguments in this environment need to
be escaped in the style of cmd.exe, not bash. (@isaacs)
TESTS
291aba7b8 make tests pass on Windows (@isaacs)
fea3a023a travis: run tests on Windows as well (@isaacs)
6.11.1 (2019-08-20):
Fix a regression for windows command shim syntax.
37db29647 cmd-shim@3.0.2 (@isaacs)
v6.11.0 (2019-08-20):
A few meaty bugfixes, and introducing peerDependenciesMeta.
FEATURES
a12341088 #224 Implements peerDependenciesMeta (@arcanis)
2f3b79bba #234 add new forbidden 403 error code (@claudiahdz)
BUGFIXES
24acc9fc8 and 45772af0d #217 npm.community#8863 npm.community#9327 do not
descend into directory deps' child modules, fix shrinkwrap files
that inappropriately list child nodes of symlink packages (@isaacs
and @salomvary)
50cfe113d #229 fixed typo in semver doc (@gall0ws)
e8fb2a1bd #231 Fix spelling mistakes in CHANGELOG-3.md (@XhmikosR)
769d2e057 npm/uid-number#7 Better error on invalid --user/--group configs.
This addresses the issue when people fail to install binary
packages on Docker and other environments where there is no
'nobody' user. (@isaacs)
8b43c9624 nodejs/node#28987 npm.community#6032 npm.community#6658
npm.community#6069 npm.community#9323 Fix the regression where
random config values in a .npmrc file are not passed to lifecycle
scripts, breaking build processes which rely on them. (@isaacs)
8b85eaa47 save files with inferred ownership rather than relying on SUDO_UID
and SUDO_GID. (@isaacs)
b7f6e5f02 Infer ownership of shrinkwrap files (@isaacs)
54b095d77 #235 Add spec to dist-tag remove function (@theberbie)
DEPENDENCIES
dc8f9e52f pacote@9.5.7: Infer the ownership of all unpacked files in
node_modules, so that we never have user-owned files in root-owned
folders, or root-owned files in user-owned folders. (@isaacs)
bb33940c3 cmd-shim@3.0.0:
9c93ac3 #2 npm#3380 Handle environment variables properly (@basbossink)
2d277f8 #25#36#35 Fix 'no shebang' case by always providing $basedir
in shell script (@igorklopov)
adaf20b #26 Fix $* causing an error when arguments contain parentheses
(@satazor)
49f0c13 #30 Fix paths for MSYS/MINGW bash (@dscho)
51a8af3 #34 Add proper support for PowerShell (@ExE-Boss)
4c37e04 #10 Work around quoted batch file names (@isaacs)
a4e279544 npm-lifecycle@3.1.3 (@isaacs):
fail properly if uid-number raises an error
7086a1809 libcipm@4.0.3 (@isaacs)
8845141f9 read-package-json@2.1.0 (@isaacs)
51c028215 bin-links@1.1.3 (@isaacs)
534a5548c read-cmd-shim@1.0.3 (@isaacs)
3038f2fd5 gentle-fs@2.2.1 (@isaacs)
a609a1648 graceful-fs@4.2.2 (@isaacs)
f0346f754 cacache@12.0.3 (@isaacs)
ca9c615c8 npm-pick-manifest@3.0.0 (@isaacs)
b417affbf pacote@9.5.8 (@isaacs)
TESTS
b6df0913c #228 Proper handing of /usr/bin/node lifecycle-path test (@olivr70)
aaf98e88c npm-registry-mock@1.3.0 (@isaacs)
This upgrade should be backward compatible.
Changelog:
- https://github.com/expressjs/session/blob/master/HISTORY.md
1.17.0 / 2019-10-10
deps: cookie@0.4.0
Add SameSite=None support
deps: safe-buffer@5.2.0
1.16.2 / 2019-06-12
Fix restoring cookie.originalMaxAge when store returns Date
deps: parseurl@~1.3.3
This upgrade should be backward compatible.
Changelogs:
- https://expressjs.com/en/changelog/4x.html
- https://github.com/expressjs/express/blob/master/History.md#4171--2019-05-25
4.17.1 - Release date: 2019-05-25
The 4.17.1 patch release includes one bug fix:
The change to the res.status() API has been reverted due to causing
regressions in existing Express 4 applications.
4.17.0 - Release date: 2019-05-16
The 4.17.0 minor release includes bug fixes and some new features, including:
The express.raw() and express.text() middleware have been added to provide
request body parsing for more raw request payloads. This uses the
expressjs/body-parser module module underneath, so apps that are currently
requiring the module separately can switch to the built-in parsers.
The res.cookie() API now supports the "none" value for the sameSite option.
When the "trust proxy" setting is enabled, the req.hostname now supports
multiple X-Forwarded-For headers in a request.
Starting with this version, Express supports Node.js 10.x and 12.x.
The res.sendFile() API now provides and more immediate and easier to
understand error when a non-string is passed as the path argument.
The res.status() API now provides and more immediate and easier to
understand error when null or undefined is passed as the argument.
Nodejs 8 will be EOLed on December 31th, 2019 (https://github.com/nodejs/Release).
This means any future Etherpad version released from 2020 on should require at
least the next LTS (10.13.0). Let's keep some margin and decide that the first
Etherpad version dropping node 8 compatibility will be 1.8.3.
Closes#3650.
Tried to simplify the document, to reduce unneded info, and to use a less
informal language.
For example, the introductory links describing git made sense 10 years ago.
Today they are not needed to understand what Etherpad is.
And mercurial was always better than git, anyways :)
The mailing list and the IRC channel seem pretty dead by now. Let's just
move everything to Github issues, which was the de facto situation anyways.
About the donation links: I am the maintainer, and I do not know the identity of
the owners of the donation links, so it is correct to remove them. The same was
done on the website three months ago:
https://github.com/ether/ether.github.com/commit/d4ef04605da5
Do not touch vendorized files (e.g. libraries that were imported from external
projects).
No functional changes.
Command:
find . -name '*.<EXTENSION>' -type f -print0 | xargs -0 sed -i 's/[[:space:]]*$//'
In preparation for next commit. I was not able to find other non-vendorized
files that were in DOS format and legitimately needed to be converted.
No functional changes.
Added pad_utils sanitization for clean and safe error handling on browsers that
do not encode the path of the URL.
Edited by muxator based on https://github.com/ether/etherpad-lite/pull/3647,
to be able to apply the patch on develop (the PR was for master), and perform
minor cleanups (mainly spurious statements).
Closes#3647.
The vendored jquery version was 1.9.1 from 2013-02-04. Let's replace it with the
most recent one from the 1.x branch (1.12.4 from 2016-05-20).
The modification in rjquery.js is needed because recent jQuery versions changed
their behaviour, and do not set themselves on the global window object.
See: https://github.com/parcel-bundler/parcel/issues/333#issuecomment-357882648
This will be the lastest jQuery 1.x version ever, because 1.x branch is
definitively EOLed (see https://github.com/jquery/jquery.com/issues/162).
This is a stopgap measure to get the latest security fixes. Going forward,
another strategy will be needed.
Closes#3640
When visiting Etherpad's home page with Chrome the "ok" button was not on the
same line as the pad name text box. On Firefox & Safari there was no problem.
Tested on Chrome 74.
Fixes#3604.
This is a non breaking change.
From the changelog (https://github.com/expressjs/session/blob/v1.16.1/HISTORY.md#1161--2019-04-11):
# 1.16.1 / 2019-04-11
- Fix error passing data option to Cookie constructor
- Fix uncaught error from bad session data
# 1.16.0 / 2019-04-10
- Catch invalid cookie.maxAge value earlier
- Deprecate setting cookie.maxAge to a Date object
- Fix issue where resave: false may not save altered sessions
- Remove utils-merge dependency
- Use safe-buffer for improved Buffer API
- Use Set-Cookie as cookie header name for compatibility
- deps: depd@~2.0.0
- Replace internal eval usage with Function constructor
- Use instance methods on process to check for listeners
- perf: remove argument reassignment
- deps: on-headers@~1.0.2
- Fix res.writeHead patch missing return value
This is just a dev dependency, so no real risks, but it's better not to scare
users.
Reported vulnerability before this change:
$ npm audit
=== npm audit security report ===
# Run npm install --save-dev nyc@14.1.0 to resolve 1 vulnerability
SEMVER WARNING: Recommended action is a potentially breaking change
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High │ Prototype Pollution │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package │ handlebars │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ nyc [dev] │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path │ nyc > istanbul-reports > handlebars │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info │ https://nodesecurity.io/advisories/755 │
└───────────────┴──────────────────────────────────────────────────────────────┘
Currently the version is exposed in a 'Server' http headers.
This commit allows to parameterize it in the settings. By defaults it is
not exposed.
Fixes#3423
The current behaviour is to show the chat bubble and hide if chat is
disabled.
Because of this, the bubble appears wrongfully for a short time.
With this PR, by default it is hidden and displayed only if chat is
enabled.
Fixes: #3088
In this way the only external call to statFile() provides an explicit value for
"dirStatLimit", and thus the initial check on "undefined" at the start of the
function could be removed (just added a comment for now).
All the configuration values can be read from environment variables using the
syntax "${ENV_VAR_NAME}".
This is useful, for example, when running in a Docker container.
EXAMPLE:
"port": "${PORT}"
"minify": "${MINIFY}"
"skinName": "${SKIN_NAME}"
Would read the configuration values for those items from the environment
variables PORT, MINIFY and SKIN_NAME.
REMARKS:
Please note that a variable substitution always needs to be quoted.
"port": 9001, <-- Literal values. When not using substitution,
"minify": false only strings must be quoted: booleans and
"skin": "colibris" numbers must not.
"port": ${PORT} <-- ERROR: this is not valid json
"minify": ${MINIFY}
"skin": ${SKIN_NAME}
"port": "${PORT}" <-- CORRECT: if you want to use a variable
"minify": "${MINIFY}" substitution, put quotes around its name,
"skin": "${SKIN_NAME}" even if the required value is a number or a
boolean.
Etherpad will take care of rewriting it to
the proper type if necessary.
Resolves#3543
Before this commit, when passed a malformed credentials.json the application
crashed with a stack dump. Now we catch the error and fail in a controlled way
(like already done for settings.json).
Example of exception we no longer throw:
MALFORMEDJSON
^
SyntaxError: Unexpected token M in JSON at position 0
at JSON.parse (<anonymous>)
at Object.reloadSettings (<BASEDIR>/src/node/utils/Settings.js:390:24)
at Object.<anonymous> (<BASEDIR>/src/node/utils/Settings.js:543:9)
at Module._compile (module.js:635:30)
at Object.Module._extensions..js (module.js:646:10)
at Module.load (module.js:554:32)
at tryModuleLoad (module.js:497:12)
at Function.Module._load (module.js:489:3)
at Module.require (module.js:579:17)
at require (internal/module.js:11:18)
ueberDB2 can return either undefined or null for a missing key, depending on
which DB driver is used. This patch changes the promise version of the API so
that it will always return null.
some code chunks previously used `async.parallel` but if you
use `await` that forces them to be run serially. Instead,
you can initiate the operation (getting a Promise) and then
_later_ `await` the result of that Promise.
If you use `await` inside a loop it makes the loop inherently serial.
If you omit the `await` however, the tasks will all start but the loop
will finish while the tasks are still being scheduled.
So, to make a set of tasks run in parallel but then have the
code block after the loop once all the tasks have been completed
you have to get an array of Promises (one for each iteration) and
then use `Promise.all()` to wait for those promises to be resolved.
Using `Array#map` is a convenient way to go from an array of inputs
to the require array of Promises.
NB1: needs additional review and testing - no abiword available on my test bed
NB2: in ImportHandler.js, directly delete the file, and handle the eventual
error later: checking before for existence is prone to race conditions,
and does not handle any errors anyway.
- removed possible issue with failing to sanitize `padName` if `padId` was also
supplied
- removed unnecessary `try` block
- simplified API and function name matching tests
Also converted the handler functions that depend on checkAccess() into async
functions too.
NB: this commit needs specific attention to it because it touches a lot of
security related code!
This patch also contains significant refactoring relating to error checking of
arguments supplied to the functions (e.g. rev) facilitated by use of `throw`
instead of nodeback errors.
1. This module was not migrated to Promises, because it is only used via
express-session, which can't actually use promises anyway.
2. all(), clear() and length() depend on the presence of the `db.forEach()`
function, which in ueberdb2 doesn't even exist.
Fortunately those three methods are optional, so I made their existence
conditional on the presence of `db.forEach`.
3. in SessionStore.clear(), replaced a call to db.db.remove() with db.remove()
Changed two occurrences of:
process.nextTick(function() {
if (fn) fn();
});
with
if (fn) {
process.nextTick(fn);
}
i.e. such that no function even gets added to the `nextTick` queue unless
there's actually a function to be called.
Extracted from Ray's work.
`getPadAccess()` (src/node/padaccess.js) is now "promise only", resolving to
`true` or `false` as appropriate, and throwing an exception if there's an
error.
The two call sites (padreadonly.js and importexport.js) updated to match.
Use real `async` instead of async.js where applicable.
The `getPluginTests()` function was never truly async anyway because it only
contains calls to synchronous `fs` modules.
Since this code can end up loaded in browsers when using client side plugins,
avoid use of ES6 syntax features such as arrow functions until MSIE support is
finally dropped.
This change is in preparation of the future async refactoring by Ray. It tries
to extract as many changes in boolean conditions as possible, in order to make
more evident identifying eventual logic bugs in the future work.
This proved already useful in at least one case.
BEWARE: this commit exposes an incoherency in the DB API, in which, depending
on the driver used, some functions can return null or undefined. This condition
will be externally fixed by the final commit in this series ("db/DB.js: prevent
DB layer from returning undefined"). Until that commit, the code base may have
some bugs.
This change extracts the grammar correction performed on the async branch,
anticipating them in a single commit. It cannot be folded with the previous
one, as it is not purely cosmetic.
This change is only cosmetic. Its aim is do make it easier to understand the
async changes that are going to be merged later on. It was extracted from the
original work from Ray Bellis.
To verify that nothing has changed, you can run the following command on each
file touched by this commit:
npm install uglify-es
diff --unified <(uglify-js --beautify bracketize <BEFORE.js>) <(uglify-js --beautify bracketize <AFTER.js>)
This is a complete script that does the same automatically (works from a
mercurial clone):
```bash
#!/usr/bin/env bash
set -eu
REVISION=<THIS_REVISION>
PARENT_REV=$(hg identify --rev "${REVISION}" --template '{p1rev}')
FILE_LIST=$(hg status --no-status --change ${REVISION})
UGLIFYJS="node_modules/uglify-es/bin/uglifyjs"
for FILE_NAME in ${FILE_LIST[@]}; do
echo "Checking ${FILE_NAME}"
diff --unified \
<("${UGLIFYJS}" --beautify bracketize <(hg cat --rev "${PARENT_REV}" "${FILE_NAME}")) \
<("${UGLIFYJS}" --beautify bracketize <(hg cat --rev "${REVISION}" "${FILE_NAME}"))
done
```
This is documented to be more performant.
The substitution was made on frontend code, too (i.e., the one in /static),
because Date.now() is supported since IE 9, and we are life supporting only
IE 11.
Commands:
find . -name *.js | xargs sed --in-place "s/new Date().getTime()/Date.now()/g"
find . -name *.js | xargs sed --in-place "s/(new Date()).getTime()/Date.now()/g"
Not done on jQuery.
The guard condition on count being non negative and < 100 used the wrong
boolean operator. In its form it was impossible.
This error was introduced in 2013, in 5592c4b0fe.
Fixes#3499
The HTTP API doesn't ever omit arguments, it always passes `undefined` for a
parameter that wasn't supplied in the request.
The functions that were simplified are:
- getRevisionChangeset()
- getText()
- getHTML()
- saveRevision()
The only function still supporting optional arguments is getPadSafe(), which is
only called from this module.
This function was simulating two overloads:
1. copy(destinationID, force, callback)
2. copy(destinationID, callback), in this case "force" would be assumed false
But all the call sites always used the version with arity 3.
Thus, we can remove that optionality and always assume that the funcion will be
called with three parameters. This will simplify future work.
Moving classes to html tag so it can be used to style other part of template depending on plugins like #users, #chat etc...
Rename plugin class with "plugin-" prefix, because there were conflicts with some plugins using the same .ep_font_color class to apply css rules
- path.exists() is no longer part of nodejs
- fs.exists() is deprecated (as of nodejs >= 8)
- checking a file for existence before using it is open to raca condition. It is
preferable to go ahead and use the file, and eventually handle the error
- we can afford two simple synchronous fs operations here
Next version will be Etherpad 1.8. As planned in #3424, we are going to require
NodeJS >=8.9.0 and npm >= 6.4.
This commit implements that change and updates documentation and scripts.
Subsequent changes will get rid of old idioms, dating back to node < 0.7, that
still survive in the code.
Once migrated to NodeJS 8, we will be able to start working on migrating the
code base from callbacks to async/await, greatly simplifying legibility (see
#3540).
Closes#3557
Until Etherpad 1.7.5, process.on('SIGTERM') and process.on('SIGINT') were not
hooked up under Windows, because old nodejs versions did not support them.
This excluded the possibility of doing a graceful shutdown of the database
connection under that platform.
According to nodejs 6.x documentation, it is now safe to do so. This allows to
gracefully close the DB connection when hitting CTRL+C under Windows, for
example.
Source: https://nodejs.org/docs/latest-v6.x/api/process.html#process_signal_events
- SIGTERM is not supported on Windows, it can be listened on.
- SIGINT from the terminal is supported on all platforms, and can usually be
generated with <Ctrl>+C (though this may be configurable). It is not
generated when terminal raw mode is enabled.
A Windows manual install has the same directory layout of a normal Unix one
(e.g. the nice symlink node_modules/ep_etherpad-lite -> ../src).
Only when running from the pre-built Windows package the directory layout is
different (e.g. src is physically copied into node_modules/ep_etherpad-lite).
The previous version of the code wrongly assumed that all Windows installs would
be run from the pre-built pakage.
In this version the path search is the same on all platform. If it fails, and we
are on Windows, there is a fallback for the specific case of the pre-built
package.
Fixes#3550
This commit vastly shortens (and simplifies) the version table within
handler/APIHandler.js by building each version's entry incrementally based off
the previous version.
The resulting table has been validated by comparing the "before" and "after"
output of the following loop on both versions of the code (albeit with an
intermediate "sort" step to account for the different insertion order)
for (let v in version) {
let m = version[v];
for (let [k, a] of Object.entries(m)) {
console.log(v, k, a);
}
}
The patch also fixes a few typos, and removes a duplicate definition of
getChatHistory which in each applicable version was defined with two different
parameter lists, but where only the second would be used.
Compatibility with IE11 regressed in 23eab79946 while working for #3488.
That commit made use of modern js syntax, not supported by IE11.
- Removed arrow functions, replaced with normal functions.
- Removed the spread operator (<...iterable>) and the "new Set()" construct,
replaced with _.uniq()
At some point IE11 compatibility will be dropped.
Ditching it now, for such a small gain, is not wise.
Fixes#3500.
This is just a dev dependency, so no real risks, but it's better not to scare
users.
Reported vulnerability before this change:
$ npm audit
=== npm audit security report ===
# Run npm update cryptiles --depth 4 to resolve 1 vulnerability
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High │ Insufficient Entropy │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package │ cryptiles │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ wd [dev] │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path │ wd > request > hawk > cryptiles │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info │ https://nodesecurity.io/advisories/720 │
└───────────────┴──────────────────────────────────────────────────────────────┘
Etherpad-lite relies on the user's browser to generate a random pad
name, but the current solution is not safe against collisions. In order
to generate unique pad names, the following modifications are made:
* use a PRNG instead of Math.random() and ensure uniform distribution
when selecting chars.
* choose the pad name length to achieve a specific number of bits of
security.
Closes: #3516
muxator