By specification [0], the if-modified-since HTTP header sent by browsers does
not include milliseconds.
Before this patch, let's say a file was generate at time:
t_real-file = 2020-03-22T02:15:53.548Z (note the fractional seconds)
When issuing a conditional request, the browser would truncate the fractional
part, and only request an if-modified-since with this contents:
t_if-modified-since = 2020-03-22T02:15:53.000Z
The minify() function would return HTTP/304 only if
t_if-modified-since >= t_real-file, but this would never be true unless, by
chance, a file was generated at XX.000Z.
This resulted in that file being minified/compressed again and resent to the
client for no reason. After this patch, the server correctly responds with
HTTP/304 without doing any computation, and the browser uses the cached file.
[0] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since
No variation in # of security vulnerabilities.
The output of `npm audit` goes from this:
found 7 vulnerabilities (5 low, 2 high) in 13881 scanned packages
7 vulnerabilities require manual review. See the full report for details.
To this:
found 7 vulnerabilities (5 low, 2 high) in 13795 scanned packages
7 vulnerabilities require manual review. See the full report for details.
This is a dev dependency. No impact in production.
After this upgrade the backend tests ("cd src; npm test") still pass.
This fixes 2 reported security vulnerabilities.
The output of `npm audit` goes from this:
found 9 vulnerabilities (7 low, 2 high) in 13707 scanned packages
run `npm audit fix` to fix 1 of them.
1 vulnerability requires semver-major dependency updates.
7 vulnerabilities require manual review. See the full report for details.
To this:
found 7 vulnerabilities (5 low, 2 high) in 13881 scanned packages
7 vulnerabilities require manual review. See the full report for details.
This is a dev dependency. No impact in production.
After this upgrade the backend tests ("cd src; npm test") still pass.
This fixes 1 reported security vulnerability.
The output of `npm audit` goes from this:
found 10 vulnerabilities (8 low, 2 high) in 13390 scanned packages
run `npm audit fix` to fix 2 of them.
1 vulnerability requires semver-major dependency updates.
7 vulnerabilities require manual review. See the full report for details.
To this:
found 9 vulnerabilities (7 low, 2 high) in 13707 scanned packages
run `npm audit fix` to fix 1 of them.
1 vulnerability requires semver-major dependency updates.
7 vulnerabilities require manual review. See the full report for details.
This fixes 107 reported security vulnerabilites.
The output of `npm audit` goes from this:
found 117 vulnerabilities (115 low, 2 high) in 13365 scanned packages
run `npm audit fix` to fix 6 of them.
1 vulnerability requires semver-major dependency updates.
110 vulnerabilities require manual review. See the full report for details.
To this:
found 10 vulnerabilities (8 low, 2 high) in 13390 scanned packages
run `npm audit fix` to fix 2 of them.
1 vulnerability requires semver-major dependency updates.
7 vulnerabilities require manual review. See the full report for details.
The changelog for npm can be read at https://github.com/npm/cli/releases
CleanCSS 3.4.19 had a Regex Denial of Service vulnerability and has to be
updated. The major version bump requires the following changes:
1. Disabling rebase is necessary because otherwise the URLs for the web fonts
become wrong;
EXAMPLE 1:
/static/css/src/static/font/fontawesome-etherpad.woff
instead of
/static/font/fontawesome-etherpad.woff
EXAMPLE 2 (this is more surprising):
/p/src/static/font/opendyslexic.otf
instead of
/static/font/opendyslexic.otf
2. CleanCSS.minify() can either receive a string containing the CSS, or an array
of strings. In that case each array element is interpreted as an absolute
local path from which the CSS file is read.
In version 4.x, CleanCSS API was simplified, eliminating the relativeTo
parameter, and thus we cannot use our already loaded "content" argument, but
we have to wrap the absolute path to the CSS in an array and ask the library
to read it by itself.
Fixes#3616.
The next commit will update a dependency in package.json. This commit only
refreshes package-lock.json recalculating it at the current date.
In this way we will be sure that any modifications in package-lock.json in the
next commit will be only due to the package.json change.
Should this commit introduce any regression, we would know for certain that one
of our dependencies is not correctly honoring their semver promises.
For some reason authorInfo is sometimes null, and therefore it is not possible
to get colorId from it.
This resulted in the following stack trace:
[2020-03-16 09:27:17.291] [ERROR] console - (node:1746) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'colorId' of null
at <BASEDIR>/src/node/handler/PadMessageHandler.js:1199:37
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async Promise.all (index 0)
at async handleClientReady (<BASEDIR>/src/node/handler/PadMessageHandler.js:1171:5)
[2020-03-16 09:27:17.291] [ERROR] console - (node:1746) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 76)
[2020-03-16 09:27:19.034] [WARN] message - Dropped message, USERINFO_UPDATE Session not ready.[object Object]
Which is due to a bug in Etherpad that we are not going to solve now.
As a workaround, when this happens, let's set the username to "Anonymous" (if
it is not already set), and colorId to the fixed value "#daf0b2". Warning
messages are written in the logs to signal this condition.
This is no definitive solution, but fixes#3612 (via a workaround).
Before this patch, visiting the read-only URL for a random pad would remove
the "Save Revision" (the "star" icon) from all the other RW pads. The only way
to make it appear again was to restart the server.
This change does not fix the underlying bug: after visiting a read only link
the "star" button would still disapper, but it is explictly reinserted via an
ad-hoc condition.
Fixes#3702
That commit (merged recently with PR #3622) was part of an effort to fix#3620,
but introduced a very bad bug that broke the cursor behaviour when pressing
space, making the program unusable.
This commit completes the revert of PR #3622 and fixes#3728.
--HG--
branch : revert-3622
By specification, when settings.allowUnknownFileEnds is true and the user tries
to import a file with an unknown extension (this includes no extension),
Etherpad tries to import it as txt.
This broke in Etherpad 1.8.0, that abruptly terminates the processing with an
UnhandledPromiseRejectionWarning.
This patch restores the intended behaviour, and allows to import as text a file
with an unknown extension (on no extension).
In order to catch the UnhandledPromiseRejectionWarning we had to use
fsp_rename(), which is declared earlier in the code and is promised based
instead of fs.rename(), which is callback based.
Fixes#3710.
Before this change, invoking a non existing API method would return an HTTP/200
response with a JSON payload {"code":3,"message":"no such function"}.
This commit changes the HTTP status code to 404, leaving the payload as-is.
Before:
curl --verbose "http://localhost:9001/api/1/notExisting?apikey=ABCDEF"
< HTTP/1.1 200 OK
< X-Powered-By: Express
[...]
{"code":3,"message":"no such function","data":null}
After:
curl --verbose "http://localhost:9001/api/1/notExisting?apikey=ABCDEF"
< HTTP/1.1 404 OK
< X-Powered-By: Express
[...]
{"code":3,"message":"no such function","data":null}
Fixes#3546.
Before this change there was always a single toolbar on the top, with both
Colibris and the legacy skin. When the screen size was reduced:
- the legacy skin would compact the icons in the toolbar (this was fine,
indeed);
- Colibris would hide some formatting icons. This would hamper the functionality
for mobile users.
After this change both the skins work in the same way, which is the following:
- when the screen gets smaller the right toolbar (the one with "export",
"timeslider", and other buttons) goes to the bottom of the screen;
- when there are many icons, the toolbar keeps all of them, and to see them the
user must drag the toolbar.
This behaviour will probably be changed before release, opting instead to show
a "+" button when there is an overflow, since this appears to be more
discoverable (see the discusison in #3697).
Do not tested with custom toolbar elements (toolbar.left and toolbar.right
configuration items in settings.json).
Fixes#3697.
Steps to reproduce (via HTTP API):
1. create a group via createGroup()
2. create a group pad inside that group via createGroupPad()
3. make that pad public calling setPublicStatus(true)
4. access the pad via a clean web browser (with no sessions)
5. UnhandledPromiseRejectionWarning: apierror: sessionID does not exist
This was due to an overlook in 769933786cea: "apierror: sessionID does not
exist" may be a legal condition if we are also visiting a public pad. The
function that could throw that error was sessionManager.getSessionInfo(), and
thus it needed to be inside the try...catch block.
Please note that calling getText() on the pad always return the pad contents,
*even for non-public pads*, because the API bypasses the security checks and
directly talks to the DB layer.
Fixes#3600.
Note by muxator:
This commit introduced a copied & modified version of the testing files
loadSettings.js and pad.js.
It's Christmas night, and we want to shipt this feature, so I merged it anyway,
adding a note in both the original and copied files so that hopefully someone
in the distant future is going to merge them back again.
This fixes some security vulnerabilites, among them an arbitrary file overwrite.
The output of `npm audit` goes from this:
found 17 vulnerabilities (15 low, 2 high) in 13344 scanned packages
run `npm audit fix` to fix 6 of them.
1 vulnerability requires semver-major dependency updates.
10 vulnerabilities require manual review. See the full report for details.
To this:
found 5 vulnerabilities (3 low, 2 high) in 13370 scanned packages
1 vulnerability requires semver-major dependency updates.
4 vulnerabilities require manual review. See the full report for details.
Changelog:
- https://github.com/npm/cli/releases
6.13.4 (2019-12-11)
BUGFIXES
320ac9aee npm/bin-links#12 npm/gentle-fs#7 Do not remove global bin/man links inappropriately (@isaacs)
DEPENDENCIES
52fd21061 gentle-fs@2.3.0 (@isaacs)
d06f5c0b0 bin-links@1.1.6 (@isaacs)
6.13.3 (2019-12-09)
DEPENDENCIES
19ce061a2 bin-links@1.1.5 Properly normalize, sanitize, and verify bin entries in package.json.
59c836aae npm-packlist@1.4.7
fb4ecd7d2 pacote@9.5.11
5f33040 #476npm/pacote#22npm/pacote#14 fix: Do not drop perms in git when not root (isaacs, @darcyclarke)
6f229f7 sanitize and normalize package bin field (isaacs)
1743cb339 read-package-json@2.1.1
6.13.2 (2019-12-03)
BUG FIXES
4429645b3 #546 fix docs target typo (@richardlau)
867642942 #142 fix(packageRelativePath): fix 'where' for file deps (@larsgw)
d480f2c17 #527 Revert "windows: Add preliminary WSL support for npm and npx" (@craigloewen-msft)
e4b97962e #504 remove unnecessary package.json read when reading shrinkwrap (@Lighting-Jack)
1c65d26ac #501 fix(fund): open url for string shorthand (@ruyadorno)
ae7afe565 #263 Don't log error message if git tagging is disabled (@woppa684)
4c1b16f6a #182 Warn the user that it is uninstalling npm-install (@Hoidberg)
The mechanism used for determining if the application is being served over SSL
is wrapped by the "express-session" library for "express_sid", and manual for
the "language" cookie, but it's very similar in both cases.
The "secure" flag is set if one of these is true:
1. we are directly serving Etherpad over SSL using the native nodejs
functionality, via the "ssl" options in settings.json
2. Etherpad is being served in plaintext by nodejs, but we are using a reverse
proxy for terminating the SSL for us;
In this case, the user has to be instructed to properly set trustProxy: true
in settings.json, and the information wheter the application is over SSL or
not will be extracted from the X-Forwarded-Proto HTTP header.
Please note that this will not be compatible with applications being served over
http and https at the same time.
The change on webaccess.js amends 009b61b338, which did not work when the SSL
termination was performed by a reverse proxy.
Reference for automatic "express_sid" configuration:
https://github.com/expressjs/session/blob/v1.17.0/README.md#cookiesecureCloses#3561.
The "io" cookie is created by socket.io, and its purpose is to offer an handle
to perform load balancing with session stickiness when the library falls back to
long polling or below.
In Etherpad's case, if an operator needs to load balance, he can use the
"express_sid" cookie, and thus "io" is of no use.
Moreover, socket.io API does not offer a way of setting the "secure" flag on it,
and thus is a liability.
Let's simply nuke it.
References:
https://socket.io/docs/using-multiple-nodes/#Sticky-load-balancinghttps://github.com/socketio/socket.io/issues/2276#issuecomment-147184662 (not totally true, actually, see above)
Revision 5879037ddc fixed a security bug, but introduced a regression, where
on page load the js console showed:
ReferenceError: require is not defined
The reason was that the fix called require('../static/js/pad_utils') to load a
module at a time when require() was still not defined.
This change anticipates the loading of require-kernel, and manually loads
pad_utils.
The fix proposed in #3670 by aaron-costello, which seemed to do the right
thing, anticipating the configuration phase of require-kernel, did not work.
It had to be declined and replaced by this (less elegant) change.
This upgrade solves the high-severity vulnerabilities regarding
https-proxy-agent that were still present in 8e6bca456f.
The output of `npm audit` goes from this:
found 29 vulnerabilities (3 low, 26 high) in 13338 scanned packages
run `npm audit fix` to fix 4 of them.
1 vulnerability requires semver-major dependency updates.
24 vulnerabilities require manual review. See the full report for details.
To this:
found 5 vulnerabilities (3 low, 2 high) in 13338 scanned packages
1 vulnerability requires semver-major dependency updates.
4 vulnerabilities require manual review. See the full report for details.
Changelog:
- https://github.com/npm/cli/releases
6.13.1 (2019-11-18)
BUG FIXES
938d6124d #472 fix(fund): support funding string shorthand (@ruyadorno)
b49c5535b #471 should not publish tap-snapshot folder (@ruyadorno)
3471d5200 #253 Add preliminary WSL support for npm and npx (@infinnie)
3ef295f23 #486 print quick audit report for human output (@isaacs)
TESTING
dbbf977ac #278 added workflow to trigger and run benchmarks (@mikemimik)
b4f5e3825 #457 feat(docs): adding tests and updating docs to reflect changes in registry teams API. (@nomadtechie)
454c7dd60 #456 fix git configs for git 2.23 and above (@isaacs)
DEPENDENCIES
661d86cd2 make-fetch-happen@5.0.2 (@claudiahdz)
6.13.0 (2019-11-05)
NEW FEATURES
4414b06d9 #273 add fund command (@ruyadorno)
BUG FIXES
e4455409f #281 delete ps1 files on package removal (@NoDocCat)
cd14d4701 #279 update supported node list to remove v6.0, v6.1, v9.0 - v9.2 (@ljharb)
DEPENDENCIES
a37296b20 pacote@9.5.9
d3cb3abe8 read-cmd-shim@1.0.5
TESTING
688cd97be #272 use github actions for CI (@JasonEtco)
9a2d8af84 #240 Clean up some flakiness and inconsistency (@isaacs)
Added `rel="noreferrer"` to automatically generated links in the main pad window
as well as the chat window.
`rel="noreferrer"` is part of the HTML5 standard. While browser support isn't
100%, it's better than nothing. Future alternative solutions with wider browser
support, such as intermediary redirect pages, are unaffected by this change.
https://html.spec.whatwg.org/multipage/links.html#link-type-noreferrer
This commit was originally part of https://github.com/ether/etherpad-lite/pull/2498
The change that implemented #3648 (7c099fef5e) was incorrect, and resulted
in disabling every user at startup.
The problem was twofold:
1. _.filter() on an object returns an array of the object's enumerable values
and strips out the keys, see: https://stackoverflow.com/questions/11697702/how-to-use-underscore-js-filter-with-an-object
To filter an object, the function that needs to be used is _.pick();
2. The logic condition on userProperties.password was plain wrong (it should
have been an AND instead of an OR).
This change corrects 1) and 2), and writes more specific logs when something
goes wrong.
Closes#3661.
This change reverts c4918efc1b, and basically negates what was done for #3396,
but aligns better with current practices in the nodejs ecosystem.
Pragmatically speaking, this will allow users, if they want, to use
npm-force-resolutions (https://github.com/rogeriochaves/npm-force-resolutions)
to manually fix security vulnerabilities.
We had a problem for that (see #3598), and - given the fragmented nature of
the nodejs ecosystem - it is reasonable to expect more issues like that one,
so it's better to be prepared.
Closes#3659.
The previous attempt to directly release 1.8.0 had to be hold back, and indeed
1.8.0 was never tagged.
Since 1.8.0 contains many changes, let's do a prerelease instead.
Closes#3660
John McLear