* Use windows.
* Fixed windows yml.
* Also upload windows build on tag to release folder.
* Added comment.
* Updated file name to be in sync with release name.
* Revert "Updated file name to be in sync with release name."
This reverts commit 2fbc26a891.
* Reverted and just rename at the end.
cc @rdelaage
on alpine, plugin command (used in src/node/utils/Abiword.js at line 31)
is not installed by defaut. Add this plugin when installing abiword.
Co-authored-by: ppom <>
* jQuery: Migrate to `.on()`, `.off()`, `.trigger()`
This avoids methods that are deprecated in newer versions of jQuery.
* jQuery: avoid `.removeAttr`, prefer `.prop`
* helper.edit: wait up to 10 seconds for ACCEPT_COMMIT
* Chat: disabled attribute is boolean
* Chat: avoid inline onclick handler to support jQuery 3.4+
* jQuery: update to version 3.6.0
* Update to 3.7
* Removed deprecated event.
* Revert change to focus on padeditor.ace
---------
Co-authored-by: webzwo0i <webzwo0i@c3d2.de>
* Use updated env var.
* Show bin and src bin.
* Use native link.
* Use link.
* Check file link.
* Use existing installation on github runner.
* Use -P.
* Use git checkout for copying the data to temp directory.
* Use rsync to copy data.
* Remove package from src.
* Use simple copy to copy the dependencies.
* Copy src folder only.
* test cov
* Added test for checking if a new pad can be created and deleted.
---------
Co-authored-by: SamTV12345 <40429738+samtv12345@users.noreply.github.com>
* SecretRotator: New class to coordinate key rotation
* express-session: Enable key rotation
* Added new entry in docker.adoc
* Move to own package.Removed fallback as Node 16 is now lowest node version.
* Updated package-lock.json
---------
Co-authored-by: SamTV12345 <40429738+samtv12345@users.noreply.github.com>
* New option to make pad names case-insensitive
fixes#3844
* fix helper.gotoTimeslider()
* fix helper.aNewPad() return value
* Update src/node/utils/Settings.js
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* remove timeout
* rename enforceLowerCasePadIds to lowerCasePadIds
* use before and after hooks
* update with socket specific test
* enforce sanitizing padID for websocket connections
- only enforce for newly created pads, to combat case-sensitive pad name hijacking
* Added updated package.json file.
---------
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Co-authored-by: SamTV12345 <40429738+samtv12345@users.noreply.github.com>
* Bumped ueberdb2 to 4.1.1
* Install only production ready dependencies.
* Added optimized Dockerfile.
* Fixed variable detection.
* Move to own variable for detecting production build.
* Use shell syntax for parameter expansion.
* Use shell as default.
* Move from deprecated request package to axios.
* Fixed package.json
* Another check.
* Fixing npm - hopefully the last.
* Remove double parsing of JSON.
* Bump bundled npm to also get rid of request in the bundled npm.
* Revert "Bump bundled npm to also get rid of request in the bundled npm."
This reverts commit b60fa4f435.
For reasons I don't understand, an activated popup was visible during
transition even though the boolean `visibility` property didn't switch
to `visible` until the end of the 0.3s transition. This prevented
input elements from getting focus until the end of the transition. Now
input elements can get focus right away.
The `ep_openid_connect` plugin needs access to session state before
authorization checks are made (to securely redirect the user back to
the start page when authentication completes). Now that the
`expressPreSession` hook exists, the rationale for moving
`preAuthorize` before the `express-session` middleware is gone.
This change undoes the following commits:
* bf35dcfc50
* 0b1ec20c5c
* 30544b564e
Prevent "TypeError: Cannot read properties of null (reading 'sheet')"
exception because google chrome can translate `<style type="text/css" title="dynamicsyntax"></style>` title attribute
This allows plugins to depend on the not-yet-released API by bumping
their `peerDependencies` to `>=1.9.0`.
IMPORTANT: v1.9.0 IS NOT RELEASED YET. I tried to bump the version to
1.9.0-alpha.0 instead, but unfortunately that doesn't satisfy
`>=1.8.6` which would break just about every plugin.
There are no guarantees about the order of execution of hook
functions, which means that a plugin's `expressConfigure` hook
function could theoretically register a handler/middleware before the
access check middleware is registered. If that happens, the plugin's
handler would run before the access check, which would be bad. Avoid
the problem by explicitly installing the `webaccess.checkAccess`
middleware before running the `expressConfigure` hook.
Send a 500 HTTP status code to the client if the session entry could
not be fetched from the database. This is useful in case the database
is busy and can't respond to the query in time. In this case we want
to abort the client connection as soon as possible.
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* Factor out plugin query.
* Make idempotent.
* Improve logging.
* Install by symlinking to a parallel directory rather than cloning
into `etherpad-lite/node_modules`.
These options are used as strings, so it doesn't make sense to default
them to a boolean value.
Note that this change has no effect due to a bug in how pad options
are processed; that bug will be fixed in a future commit.
It doesn't make sense to override the browser's language with `en-gb`
by default.
Note that this change has no effect due to a bug in how pad options
are processed; that bug will be fixed in a future commit.
The Node.js 14 slim image has quite a few vulnerabilities, and I have
tested the latest slim image. It works just fine.
When installing plugins, `--legacy-peer-deps` is passed to npm because
npm v7 (which comes with Node.js v16, the current LTS) changed how
peer deps are handled. The new behavior is incompatible with how
plugins have historically been installed.
This puts global state change logic with the rest of the global state
management logic. This also makes it possible to create temporary Pad
objects without triggering plugin actions.
These files cause problems with Docker images and read-only
directories/mounts, and they have dubious value (any install-time
setup should instead be done at startup).
This change
- removes instructions about commit headers that nobody follows,
- links to useful resources for first-time contributors,
- simplifies some text, and
- hides all text inside <!-- -->.
The old "switch to pad" logic looked buggy, and it complicates pad
initialization. Forcing a refresh after importing an `.etherpad` file
isn't much of a UX downgrade.
This pulls in newer versions of some database drivers which silences
some `npm audit` security warnings.
This also adds support for PostgreSQL connection strings.
Use MutationObserver to detect if a saveProgress event was received,
which will trigger an animation.
Before this, `helper.admin$('#response').is(':visible')` was true
after the page loaded and before clicking the Save button, so there
was a possibility that after clicking Save, but before sending the
socketio message to the server, the visibility is checked and returns
true, so the page gets reloaded before the changed settings have been
saved.
Rationale:
* Clearing out `src/node_modules` is unlikely to bring future
success.
* If there is an error, it's better to leave the filesystem alone so
that the user can investigate the cause.
* Deleting the directory on error is a surprising behavior.
The comment "head and body had been removed intentionally" implies
that the tags were causing some sort of problem, but the commit that
removed them (57075d1545) didn't provide
any rationale. I'm assuming it was a mistake.
This avoids null dereference if a buggy caller calls
`toggleDropDown('none')` before `init()`. (Ideally the caller would be
fixed, but this is not always feasible.)
Cheerio provides jQuery-like objects but they wrap DOM Node-like
objects that are not 100% API compatible with the DOM spec. Because of
this, contentcollector, which is used in browsers and in Node.js
during HTML import, has until now needed to support two different
APIs. This commit modifies HTML import to use jsdom instead of cheerio
and simplifies contentcollector.
ace.js: removed the role 'application' from innerDocument.body. JAWS
do not read any text from the edit lines if this role is set.
domline.createDomLine: to give JAWS the ability to read the lines
correctly, it is required to set the attribute 'aria-live' to
'assertive'.
The `Node.nextSibling` property returns the next Node, not the next
Element. If whitespace, an HTML comment, or any other type of
non-Element Node is ever introduced between the Elements then
`.nextSibling` no longer returns the desired Element. Switching to
`Element.nextElementSibling` would work, but finding the Elements by
ID is more readable and future-proof.
* Define utility functions above their use to silence lint warnings.
* Use `.css()` instead of `.attr('style')` to manipulate style.
* Pass an object to `.attr()` rather than call once per attribute.
* Take advantage of chaining.
* Inline unnecessary `padUrl` variable.
* Delete some unnecessary comments.
It is possible for the stats to be read before the
`expressCreateServer` hook is called (in particular: when there is an
error during startup), which is when the `socketio` variable is set.
Check for non-null `socketio` before attempting to count the number of
socket.io connections.
Readability is increased by explicitly checking if jquery/sendkeys was
already loaded before evaluating it in the context of ace_inner and the
enclosing container (pad.html). Note that sendkeys is no longer
evaluated in the context of ace_outer, as this isn't needed
Also removes some IE 8/9 legacy code
When settings.useMonospaceFontGlobal is set to `true`, it sets the default
font to 'monospace'. This font seems to have been removed in
a5164dad43.
This commit sets the default font to "RobotoMono" which is a valid
option.
Tested in a Docker environment, setting `PAD_OPTIONS_USE_MONOSPACE_FONT`
to `true`
Signed-off-by: Xavier Mehrenberger <xavier.mehrenberger@gmail.com>
Using `.innerHTML` to create a `<script>` element does create a DOM
node, but the script is not actually executed. Fortunately, creating a
DocumentFragment does cause the script to execute.
* cd to top-level Etherpad directory is now more robust.
* Only attempt to update packages whose names begin with `ep_`.
* Don't create `package-lock.json`.
* Improve logging.
* Improve error handling.
This makes core and plugin tests consistent with each other, makes it
possible to `require()` relative paths in spec files, simplifies the
code somewhat, and should make it easier to move away from
require-kernel.
Also:
* Wrap plugin tests inside a `describe()` that contains the plugin
name to make it easier to grep for a plugin's tests and for
consistency with core tests.
* Add "<core>" to the core test descriptions to make it easier to
distinguish them from plugin tests.
Before this commit, webaccess.checkAccess saved the authorization in
user.padAuthorizations[padId] with padId being the read-only pad ID,
however later stages, e.g. in PadMessageHandler, use the real pad ID for
access checks. This led to authorization being denied.
This commit fixes it by only storing and comparing the real pad IDs and
not read-only pad IDs.
This fixes test case "authn user readonly pad -> 200, ok" in
src/tests/backend/specs/socketio.js.
readonly paste links should be readable even if authentication is turned
on, as long as the user provides valid login data.
This test currently fails.
Also test that readonly paste IDs can be exported under the same
condition, which currently succeeds.
This has a few benefits:
* It's more readable: It's easier for a user of the function to know
that they should use `await` when calling the function.
* Stack traces are more useful.
* Some code (e.g., the async npm package) uses introspection to
determine if a function is `async` vs. takes a callback.
Benefits of this change:
* It avoids race conditions with tests that clear cookies.
* Any attempt to get or set a value before `init()` is called will
throw an error, ensuring the API is used properly.
* Improved readability: It's easier to understand what the
`pad.noCookie` check is doing.
* Rename "current"/"other" to "user0"/"user1".
* Delete unnecessary `_createTokenFor*` functions.
* Rename helper functions to remove unnecessary leading underscore
and for brevity.
* Use jQuery's `.attr()` to build the second iframe.
* Use js-cookie to manipulate the token cookie.
* Don't attempt to set the token cookie if the pad isn't loaded.
* Use the token generated by the pad.
* Only clear the token cookie at path=/.
Move server message queue processing out of `handleUserChanges()` for
the following reasons:
* Fix a race condition: Before this change the client would stop
processing incoming messages and stop sending changes to the
server if a `NEW_CHANGES` message arrived while the user was
composing a character and waiting for an `ACCEPT_COMMIT` message.
* Improve readability: The `handleUserChanges()` function is for
handling changes from the local user, not for handling changes
from other users.
* Simplify the code.
Reusing the same op object for each iteration can result in very weird
behaviors because previously yielded op objects will get a surprise
mutation.
It is unclear why the code was written to reuse the same object. There
was no comment, nor is there a commit message providing rationale (it
has behaved this way since the very first commit). Perhaps the objects
were reused to improve performance (fewer object allocations that need
to be garbage collected). I do expect this change to reduce
performance somewhat, but not enough to warrant reverting this commit.
Safari takes a while to initialize `document.styleSheets`, which
results in a race condition when loading the pad. Avoid the race
condition by accessing the CSSStyleSheet objects directly from the
HTMLStyleElement DOM objects.
* add more endpoints that do not need a session
* Update src/node/hooks/express/webaccess.js
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* Update src/node/hooks/express/webaccess.js
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Co-authored-by: John McLear <john@mclear.co.uk>
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
This change makes no visual difference right now, but will matter (for
reasons I don't understand) once we change `ace.js` to build the
iframes by constructing elements in JavaScript (vs. writing HTML).
* Add the class "pad" to the `<html>` tag in `pad.html` (the outer
iframe's parent).
* Change the CSS selector that refers to the `<html>` tag in
`pad.html` from `html:not(.inner-editor)` to `html.pad`.
* Change the class name of the outer iframe's `<html>` tag from
"inner-editor" to "outer-editor".
* Update CSS rules to use the new class name.
* tests: CI of updating from master > this commit.
In response to cypress eslint I thought I'd put some CI testing for if a PR might break automated upgrading.
Matrix usage is probably overkill.
* Update major-version-git-pull-update.yml
* Name...
* include a front end test
* fix pathing
* Clarity on what's happening
* Update .github/workflows/major-version-git-pull-update.yml
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* Update .github/workflows/major-version-git-pull-update.yml
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* Update .github/workflows/major-version-git-pull-update.yml
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Now one can create an `etherpad` user and group on the host system and
set the container's UID and GID to match:
adduser --system --group etherpad &&
uid=$(id -u etherpad) &&
gid=$(id -g etherpad) &&
docker build --build-arg EP_UID="${uid}" --build-arg EP_GID="${gid}" .
This ensures that files created by user `etherpad` inside the
container are owned by user `etherpad` outside the container.
This reverts commit a17f9bf3cf, which
caused a mysterious bug with the line numbers. Revert to avoid
blocking a new release while I figure out the bug.
There are two main benefits:
* HTML is no longer printed in the startup debug logs.
* `require()` is no longer called on client-side files. This
eliminates "Failed to load <file> for <plugin>: ReferenceError:
window is not defined" errors when users visit
`/admin/plugins/info`.
Constructing a relative pathname on Windows is problematic because the
two absolute pathnames might be on different drives (or UNC paths).
Use `path.resolve()` instead of `path.join()` where appropriate to
avoid the need to construct a relative path.
The intention of the deleted code was to reduce the number of fetches,
but it only saved a single fetch due to implementation flaws. The
right way to reduce the number of fetches is to use a bundling
technology such as webpack, and this change makes it easier to do so.
This isn't strictly necessary right now, but will become
necessary (due to a Safari quirk) when we change to building the
iframes programmatically (vs. the current `document.write()`
approach).
* tests: Frontend test Windows ZIP
This PR introduces Frontend testing within Github actions!
We're depending a lot on saucelabs recently and that's fine but sometimes we just want to quickly do a frontend simple test on a weird environment (IE windows build) so this PR solves that problem.
Things to note.
It still builds the windows .zip if the cypress tests fail.
It does not add any heavy deps to Etherpad as cypress must be installed in CI.
Cypress is responsible for running the Etherpad instance.
It's up to us how much we use this or not, I know it introduces a bunch of technical debt but I tried to keep that a minimum by compartmentalizing things and documenting where required.
* Update .github/workflows/windows-zip.yml
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* remove timeouts
* Move folder structure up a level
* Update windows-zip.yml
* Update test.js
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Outputs a list of pluginnames and email address for maintainers to contact. Useful for me to bump folks to maintain there stuff and stop it getting stale :)
* Lint functions
* Fix assignment of `settings.minify`
* Use a for loop to avoid copied code for the `minify = true` and
`minify = false` cases
* Put each resource fetch into its own test case
* Check for 200 status code
* Use `.expect()` to check header value
* Use `.expect(fn)` instead of `.then(fn)`
* Windows CI Installer
This PR introduces CI builds of a windows installer(using NSIS) .
It builds an executable that installs Etherpad and runs it.
There are obvious steps to make once this has been merged. But I'd suggest on each release we include both the .zip and the .exe and allow users to have a portable zip or an installed executable.
https://github.com/ether/etherpad_nsis
This was a relatively rushed project (4 hours) and I didn't want to spend any more time on it so it will need a foster parent to maintain it :)
props to @joncloud for https://github.com/joncloud/makensis-action-test and the nsis team that while have a horrible UX make relatively easy to use and rapid tools.
Note for review: I'm using linux to build the windows executable, this may need to be reviewed and we might want to switch to Windows if we can confirm building on linux causes a problem.
* CI: Use Windows to build the .zip
* docs: fix links from TOC to Headings
* docs: Styling
Just a little modernisation of the appearance of the documentation
* Update src/bin/doc/package.json
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Firefox 52 has issues with rendering SVG animations which caused random tests to fail. Less than 2% of total Firefox users now use Firefox 52 so we're safe to drop testing for it.
The testing approach was redone to fix numerous issues:
* Even if the tests had been working, none of them would have caught
https://github.com/ether/etherpad-lite/issues/4808 because they
didn't exercise the client-side import logic. Now they do.
* Follow-up logic was not in the `helper.waitFor()` callback like it
should have been. Now the code uses `async` and `await` to ensure
proper execution order.
* All `$.ajax()` calls used `async: false`. Now they're properly
asynchronous.
* The `helper.waitFor()` condition callbacks threw instead of
returning false.
* The string comparisons didn't allow for different attribute
order (e.g., `<ol start="1" class="list-number1">` vs. `<ol
class="list-number1" start="1">`). Now `Node.isEqualNode()` is
used to reduce fragility. (`Node.isEqualNode()` is not perfect, so
the tests are still a bit fragile: If class names or style strings
are in a different order then `Node.isEqualNode()` will return
false even if the nodes are semantically equivalent.)
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* CI: Leave log level at INFO for frontend tests
* CI: Disable frontend admin tests for non-admin workflow
* CI: Disable import/export rate limiting for frontend tests
* tests: fix importexport tests
The testing approach was redone to fix numerous issues:
* Even if the tests had been working, none of them would have caught
https://github.com/ether/etherpad-lite/issues/4808 because they
didn't exercise the client-side import logic. Now they do.
* Follow-up logic was not in the `helper.waitFor()` callback like it
should have been. Now the code uses `async` and `await` to ensure
proper execution order.
* All `$.ajax()` calls used `async: false`. Now they're properly
asynchronous.
* The `helper.waitFor()` condition callbacks threw instead of
returning false.
* The string comparisons didn't allow for different attribute
order (e.g., `<ol start="1" class="list-number1">` vs. `<ol
class="list-number1" start="1">`). Now `Node.isEqualNode()` is
used to reduce fragility. (`Node.isEqualNode()` is not perfect, so
the tests are still a bit fragile: If class names or style strings
are in a different order then `Node.isEqualNode()` will return
false even if the nodes are semantically equivalent.)
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* code tidy up: always evaluates
* tidy up: is always true
* tidy up: remove unused code
* always true/false variables
* unused variable
* tidy up: remove unused code in caretPosition.js
* for squash: Revert "tidy up: remove unused code in caretPosition.js"
The `if` condition was previously always true, so the body should be
preserved. If the body is preserved, other logic can be deleted. I
opened PR #4845 to clean it all up.
This reverts commit 75b03e5a7d.
* for squash: simplify
* for squash: Explain that the getter is used for its side effects
It's very weird to call a getter without using its return value. Add a
comment explaining why this is done so that the reader doesn't get
confused.
* for squash: Revert "tidy up: remove unused code"
The exception test was the purpose of the code.
This reverts commit 85153b1676.
* for squash: Log the tsort results
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* pluginfw: Warn plugins on missing plugin
Add functionality to console.warn when a plugin is missing. This will help admins know when people are trying to use plugins that are missing. Resolves https://github.com/ether/etherpad-lite/issues/4730
* pluginfw: importing .etherpad can notify admins of missing plugins
Extending .etherpad imports to notify admins if a missing plugin is present
* Update ImportEtherpad.js
This also makes the full line number element clickable to ensure a positive UX for the ``?lineNumber`` URL endpoint. It also makes it more obvious that a click action can happen based on the hover.
Make line numbers stick to baseline of first line of wrapped content and editor lines with increased line hieght.
Make it compatible with ep_author_neat
Due to a recent release that wasn't functioning properly this CI will help us catch the majority of Microsoft Node Quirks before they make it into a release.
This MR introduces a docker build variable `INSTALL_ABIWORD`. When set
to any value other than `0`, ABIWORD is installed in the resulting
docker container, enabling the possibility to configure ABIWORD in
settings.json.docker or via ENV VAR `ABIWORD` for exporting to
DOC/PDF/ODT.
Documentation is included inline and in the docker markdown file.
This fixes a bug introduced in commit
b711ff6acf. Some time between when that
commit was originally written and when it was merged a round of
linting had converted the function from a regular function to an arrow
function because `this` was never in the body of the function. When I
rebased the commit, which introduced `this` to the body, I didn't
catch the error.
* tests: Restore `runnerBackend.sh`
`runnerBackend.sh` was deleted in commit
7dae5e3db8 but plugins still need it
until their GitHub workflow definitions have been updated.
Co-authored-by: John McLear <john@mclear.co.uk>
This makes it possible to change the rate limiter settings via
`/admin/settings` or by modifying the appropriate settings object and
reinvoking the hook.
I can't see any reason this would be necessary, and it appears to not
behave as intended (`scroll.scrollWhenPressArrowKeys()` is not invoked
after a continuously held arrow key is finally let up).
Patch-specific release branches should never diverge from the tag, so
they serve no useful purpose. (If they do diverge, which some did
before I deleted them all, what does it mean? Are we going to move the
tag in the future? It's just too confusing.)
In the future we might want to do major- or minor-specific
branches (e.g., `release/1` or `release/1.8`), but only if we want to
maintain old releases. For example, if 2.0 is a major release that
doesn't work with plugins designed for 1.x we might want to maintain a
`release/1` branch that continues to get bugfixes while the bulk of
new work continues to land on `develop`. If we do decide to maintain
old releases we'll need a new set of release scripts (or edit the
`release.js` script on the `release/1` branch).
Express v4.x does not check to see if a Promise returned from a
middleware function will be rejected, so explicitly pass the Promise
rejection reason to `next()`.
We can revert this change after we upgrade to Express v5.0.
See https://expressjs.com/en/guide/error-handling.html for details.
I'm not sure how these tests ever worked. I guess some version of
Node.js and npm come pre-installed on the ubuntu-latest images?
I would have prefered to use Node.js v10 because that is our current
minimum supported version, but we have a surprising number of tests
that don't work on Node.js v10 (mostly due to `assert.match()`, which
was added in Node.js v12).
Before, an unhandled rejection or uncaught exception during startup
would cause `exports.exit()` to wait forever for startup completion.
Similarly, an error during shutdown would cause `exports.exit()` to
wait forever for shutdown to complete. Now any error during startup or
shutdown triggers an immediate exit.
Logging verbosity of the openapi handlers was turned down so GitHub
should be happier with INFO now. This makes it easier to troubleshoot
problems.
This reverts commit b98aaf4904.
The settings commitRateLimiting.duration and commitRateLimiting.points
were not available in the settings.json.docker file, and therefore it
was not possible to override their values via environment variables.
Now, they can be overridden by setting the following env vars:
* commitRateLimiting.duration: COMMIT_RATE_LIMIT_DURATION
* commitRateLimiting.points: COMMIT_RATE_LIMIT_POINTS
`waitForPromise()` should always be used with `await` (either directly
or with a later `await` on the returned Promise). In this case,
the condition should be immediately true so `waitForPromise()` is not
the right tool here.
* Delete the unused `optDoNow` parameter from `pendingInit()`.
* Move the `setAuthorInfo()` 1st parameter check out of the wrapper
and in to the `setAuthorInfo()` function itself.
All of the tests in this file are commented out so this file does
nothing. We can uncomment the code and clean it up, but the approach
taken in these tests will never work: For security reasons, browsers
do not allow synthetic key events to perform the default
behavior (such as moving the carent when an arrow key is pressed).
There are two ways to test responses to navigation keys:
* Use WebDriver to create "genuine" keyboard events.
* Suppress the default behavior and implement caret movement
ourselves. This is tremendously complicated, especially arrow
up/down.
Advantages of a top-level `package.json`:
* It prevents npm from printing benign warnings about missing
`package.json` whenever a plugin is installed.
* Semantically, it is "the right thing to do" if plugins are to be
installed in the top-level directory. This avoids violating
assumptions various tools make about `package.json`, which makes
it more likely that we can easily switch to a new version of npm
or to an npm alternative.
Disadvantages of a top-level `package.json`:
* Including a dependency of `ep_etherpad-lite@file:src` in the
top-level `package.json`, which is required to keep npm from
deleting the `node_modules/ep_etherpad-lite` symlink each time a
package is installed, drastically slows down plugin installation
because npm recursively walks the ep_etherpad-lite dependencies.
* npm has a horrible dependency hoisting behavior: It moves
dependencies from `src/node_modules/` to the top-level
`node_modules/` directory when possible. This causes numerous
mysterious problems, such as silent failures in `npm outdated` and
`npm update`, and it breaks plugins that do
`require('ep_etherpad-lite/node_modules/foo')`.
Right now, with npm v6.x, eliminating the disadvantages is far more
valuable than keeping the advantages. (This might change with npm
v7.x.)
For a long time there was no top-level `package.json` and it worked
fairly well, although users were often confused by npm's benign
warnings. The top-level `package.json` was added because we needed a
place to put ESLint config for the stuff in `bin/` and `tests/`, and
we wanted the advantages listed above. Unfortunately we were unaware
of the disadvantages at the time. The `bin/` and `tests/` directories
were moved under `src/` so we no longer need the top-level
`package.json` for ESLint.
An alternative to a top-level `package.json`: Create
`plugins/package.json` and install all plugins in `plugins/`. If
`plugins/package.json` has a dependency of
`ep_etherpad-lite@file:../src` then plugin installation will still be
slow (npm will still recursively walk the dependencies in
`src/package.json`) but it should avoid npm's nasty dependency
hoisting behavior. To avoid slow plugin installation we could create a
lightweight `etherpad-pluginlib` package that Etherpad plugins would
use to indirectly access Etherpad's internals. As an added bonus, this
intermediate package could become an adaptor that provides a stable
interface to plugins even when Etherpad core rapidly evolves.
Also add symlinks from the old `bin/` and `tests/` locations to avoid
breaking scripts and other tools.
Motivations:
* Scripts and tests no longer have to do dubious things like:
require('ep_etherpad-lite/node_modules/foo')
to access packages installed as dependencies in
`src/package.json`.
* Plugins can access the backend test helper library in a non-hacky
way:
require('ep_etherpad-lite/tests/backend/common')
* We can delete the top-level `package.json` without breaking our
ability to lint the files in `bin/` and `tests/`.
Deleting the top-level `package.json` has downsides: It will cause
`npm` to print warnings whenever plugins are installed, npm will
no longer be able to enforce a plugin's peer dependency on
ep_etherpad-lite, and npm will keep deleting the
`node_modules/ep_etherpad-lite` symlink that points to `../src`.
But there are significant upsides to deleting the top-level
`package.json`: It will drastically speed up plugin installation
because `npm` doesn't have to recursively walk the dependencies in
`src/package.json`. Also, deleting the top-level `package.json`
avoids npm's horrible dependency hoisting behavior (where it moves
stuff from `src/node_modules/` to the top-level `node_modules/`
directory). Dependency hoisting causes numerous mysterious
problems such as silent failures in `npm outdated` and `npm
update`. Dependency hoisting also breaks plugins that do:
require('ep_etherpad-lite/node_modules/foo')
For some reason strings are sometimes passed to `findUnmet()`, which
is obviously unexpected given the way the code is written. Rather than
figure out why strings are passed and how to safely avoid passing
strings, just return early. The net effect is the same, but returning
early avoids setting a property on a string, which is prohibited in
strict mode.
Benefits of `callHookFnSync()` and `callHookFnAsync()`:
* They are a lot more forgiving than `hookCallWrapper()` was.
* They perform useful sanity checks.
* They have extensive unit test coverage.
* They make the behavior of `callFirst()` and `aCallFirst()` match
the behavior of `callAll()` and `aCallAll()`.
Define states and use them to properly handle multiple calls to
`start()`, `stop()`, and `exit()`. (Multiple calls to `exit()` can
happen if there is an uncaught exception or signal during shutdown.)
This should also make it easier to add support for cleanly restarting
the server after a shutdown (for tests or via an `/admin` page).
* lint: skin-variants
* for squash: Fix attachment of event listener
Before this PR the statement was outside the function. I'm assuming
the move into the function body was accidental, so move it back out.
* for squash: Preserve order of function calls
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
There are some problems with nyc:
* The coverage numbers aren't useful in our case because most of the
code is executed outside the test process (the test code is mostly
API client logic).
* nyc messes with line numbers, which makes it much harder to debug
problems.
* We're seeing frequent SIGABRT crashes while nyc is printing the
results table. I'm not sure if nyc is the cause of the crashes, or
if it's making a race condition worse, or if the crashes have
nothing to do with nyc, but we don't lose much by removing it so
we might as well see if the crash frequency improves.
This makes it much easier to see why a test is failing. Before, a
`helper.waitFor()` failure would simply cause the test to time out.
Now an exception is displayed.
Before this change, the `author` attribute was silently discarded
during `.map()` iteration and the name of the attribute to remove was
included twice with two different values.
Before this commit, the callback passed to `.map()` during attribute
removal was a normal function, not an arrow function. This meant that
the value of `this` in the function body depended on how the callback
was invoked. In this case, the callback was invoked without any
explicit context (it was not called as a method, nor was it called via
`.call()`, `.apply()`, or `.bind()`). Without any explicit context,
the value of `this` depends on strict mode. Currently the function is
in sloppy mode, so `this` refers to the "global this" object (a.k.a.,
`window`). It doesn't make sense for the callback to reference
`window.author`, so I'm assuming the previous behavior was a bug.
Now the function is an arrow function, so the value of `this` comes
from the enclosing lexical context, which in this case is the
AttributeManager object. I believe that was the original intention.
This makes it possible for plugin backend tests to do
`require('ep_etherpad-lite/tests/backend/common')` to access the API
key (among other things).
Eventually we probably should reverse these (move `tests/` to
`src/tests/` and make `tests/` a symlink to `src/tests/`) and move
`bin/` to `src/bin/` so that we can avoid the top-level `package.json`
mess.
The `name` property is only available on cheerio's Element-like
objects; DOM Element objects do not have a `name` property. Switch to
`dom.tagName()` to fix the logic for browsers.
The `parent` property is only available on cheerio's Node-like
objects; DOM Node objects do not have a `parent` property. Switch to
the `parentNode` property so that the code works in browsers as well
as cheerio.
Before, the hook always ignored the return values provided by the hook
functions. Now the hook functions can change the text by either
returning a string or setting `context.text` to the desired value.
Also drop the `styl` and `cls` context properties. They were never
documented and they were always null.
In the DOM, `.children` only includes children that are Element
objects. In cheerio 0.22.0, `.children` includes all child Nodes, not
just Elements. Use `dom.numChildNodes()` and `dom.childNode()` so that
browsers behave the same as cheerio.
`for..in` iterates over inherited properties, which is almost never
desired. In most cases there aren't any inherited enumerable
properties so it's not that big of a deal, but in the case of
HTMLCollection it's very bad because it iterates over every entry
twice (once by numerical index and once by name) plus it includes the
`length` property in the iteration.
The `attribs` property is only available on cheerio's Element-like
objects; DOM Element objects do not have an `attribs` property. Switch
to `dom.nodeAttr()` to fix the logic for browsers.
`describe()` is meant to be used by independent tests, but the tests
in this file are not independent. Add a higher-level `describe()` call
and delete all of the `describe()` calls that wrap a single test.
Various tidy up and linting of contentcollector.js and domline.js.
3 Tests disabled which are not due to be covered.
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Removes ep_page_view plugin from the installation instructions in README.md to aid new users installing etherpad-lite.
According to the plugin directory at https://static.etherpad.org/index.html, the plugin is defunct as it is now part of etherpad-lite core and when I tried installing etherpad with the plugin (following the instructions), its installation resulted in an error every time I opened a pad.
* bugfix, lint and refactor all bin scripts
* for squash: throw Error(message) rather than log(message); throw Error()
* for squash: Exit non-0 on unhandled Promise rejection
Many of the recent lint changes have converted normal functions to
async functions, and an error thrown in an async function does not
cause Node.js to exit by default.
* for squash: fix `require()` paths
* for squash: remove erroneous `Object.keys()` call
* for squash: fix missing `continue` statements
* for squash: Fix HTTP method for deleteSession
* for squash: delete erroneous throw
Throw is only for errors, not successful completion.
* for squash: redo migrateDirtyDBtoRealDB.js to fix async bugs
* for squash: fix erroneous use of `for..of`
* for squash: Add line break between statements
* for squash: put closing paren on same line as last arg
* for squash: Move `log()` back up where it was
to minimize the diff to develop
* for squash: indentation fixes
* for squash: typo fix
* for squash: wrap long lines
* for squash: use `util.callbackify` to silence promise/no-callback-in-promise warning
* for squash: use double quotes to improve readability
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* remove IE and add strict headers
* linting: kids are back, need to stop for today
* linting: farbtastic fix
* lint: more lint fixes
* more lint fixes
* linting: sub 100 errors
* comments where I need help
* ready to be helped :)
* small fixes
* fixes
* linting: all errors resolved
* linting: remove note to self
* fix as per nulli/wezz000li suggestion
* fix as per nulli/wezz000li suggestion
* resolve merge conflicts
* better use if to silence eslint
* Use `for..of` with `Object.keys` instead of `for..in`
* lint: move setSelection to before call
Co-authored-by: webzwo0i <webzwo0i@c3d2.de>
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
This makes it possible to check plugins that were installed by
symlinking into `node_modules/` like this:
git clone git@github.com:ether/etherpad-lite.git
git clone git@github.com:ether/ep_example.git
cd etherpad-lite
npm i ep_example@file:../ep_example
node ./bin/checkPlugin.js ep_example
Rather than check for modifications and untracked files in one
command, use two commands: one for modifications and one for untracked
files. This makes the error messages easier to understand, and it
allows us to include `git status`-like output in the modifications
error message.
This will make the pages gracefully handle HTTP server restart events,
which happen whenever a plugin is installed or uninstalled via the
`/admin/plugins` page.
* lint: collab-client
* Undo incorrect lint fixes
These will be re-fixed in a future commit.
* Properly fix guard-for-in error
* Properly fix prefer-rest-params errors
* Move some code back to where it was
Moving the code makes it hard to review the diff.
* Delete DISCONNECT_REASON case
Someone reading the code won't understand what "used to handle
appLevelDisconnectReason" means until they dig through the Git
history. Given the server never sends messages of type
DISCONNECT_REASON anyway, just delete the case.
* Refine lint fixes
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Squashed changes from rhansen@rhansen.org:
* Move code back to where it was. (It's easier to review changes
when the code isn't moved. This causes some no-use-before-define
warnings to reappear, but those are just warnings.)
* Move eslint-disable comment to same line
* Use `window.clientvars` to resolve no-global-assign
* Undo changes that aren't about fixing lint errors
* lint: pluginfw tsort.js
* Don't comment out the `console.log()` call
Disabling the log message is out of scope for the pull request.
* Put const and let on separate lines
* Convert `tsort` from function to arrow function
ESLint doesn't complain about this due to a bug in
prefer-arrow/prefer-arrow-functions rule:
https://github.com/TristonJ/eslint-plugin-prefer-arrow/issues/24
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* fix accidental write to global variable
properly show pending tests
log test name in suite
better log output for received/expected strings
* cc tests: enable second nestedOL test
* ignore the head tag on import
Missing await in call to this._pad.getInternalRevisionAText(rev). Function returns a promise. This bug breaks the createDiffHTML API call (how I discovered it).
These characters are in the RFC3986 reserved set.
These characters are added to the set of characters that cannot be the
last character of a URL to avoid mislinkification.
It should be the client's responsibility to handle null name or color.
In the case of author names, passing null to the client allows users
to fill in the names of other users (via a suggestUserName
CLIENT_MESSAGE).
When a new client opens a socket.io connection and sends a
CLIENT_READY message, Etherpad sends the new client a bunch of
USER_NEWINFO messages, one per other user already connected to the
pad. When iterating over the other users, filter out those without an
author ID or missing from the global authors database.
This makes the strings easier to read, and it simplifies `append()`.
Also fix some lint errors:
* Use `const` instead of `var`.
* Convert `append()` to an arrow function.
* Wrap long lines.
https://github.com/ether/etherpad-lite/pull/4516 was accidentally
squash-merged when it should have been rebased and merged (or regular
merged). This merge brings in the separate commits so that the changes
can be easily reviewed in the future.
eslint-config-etherpad 1.0.11 changed the comma-dangle rule to
prohibit trailing commas for function arguments. See:
673ab07acf
Re-run the automated fixes to apply the rule change.
This also fixes a few lint issues in changes that were made after
`eslint --fix` was originally run.
* lint: Bump eslint-config-etherpad to 1.0.11
* lint: Rerun `eslint --fix` to nuke trailing function call commas
eslint-config-etherpad 1.0.11 changed the comma-dangle rule to
prohibit trailing commas for function arguments. See:
673ab07acf
Re-run the automated fixes to apply the rule change.
This also fixes a few lint issues in changes that were made after
`eslint --fix` was originally run.
Normally I would let `eslint --fix` do this for me, but there's a bug
that causes:
const x = function ()
{
// ...
};
to become:
const x = ()
=> {
// ...
};
which ESLint thinks is a syntax error. (It probably is; I don't know
enough about the automatic semicolon insertion rules to be confident.)
Travis placed an unnecessary breaking restriction on our tests and failed to respond within 72 hours to our complaint. This has forced us to introduce Github Actions to manage our testing. This is hopefully a temporary measure while Travis either gets itself together or we find a non-Github requirement.
* Add commentary explaining why things are done the way they are.
* Delete steps that were added for debugging.
* Pass `--no-save` when installing `ep_etherpad-lite`.
* caching_middleware: fix gzip compression not triggered
* packages: If a client sets `Accept-Encoding: gzip`, the responseCache will
include `Content-Encoding: gzip` in all future responses, even
if a subsequent request does not set `Accept-Encoding` or another client
requests the file without setting `Accept-Encoding`.
Fix that.
* caching_middleware: use `test` instead of `match`
* add tests
* make code easier to understand
* make the regex more clear
* Avoid bashisms.
* Simplify `sed` of `settings.json`.
* Wrap long lines.
* Define and use the conventional log functions.
* Quote variable expansions.
`tests/frontend/travis/runner.sh` transforms `settings.json.template`
and overwrites `settings.json`, so creating `settings.json` doesn't
have any effect. Change the Travis setup to mutate
`settings.json.template` instead of `settings.json`.
* Fix bad paren placement in `/javascript` handler
This fixes a bug introduced in commit
ed5a635f4c.
* add regression test for #4495
* Move `/javascript` test to `specialpages.js`
Co-authored-by: webzwo0i <webzwo0i@c3d2.de>
Some authentication plugins use the users defined in the `users`
object but ignore the `password` and `hash` properties.
This change deletes all of the filtering logic, including the logic
that filters out users that have both `password` and `hash` properties
defined. I could have kept that check, but decided to remove it
because:
* There's no harm in defining both `hash` and `password`.
* Allowing both makes it easier to transition from one scheme to
another.
* It's fewer lines of code to maintain.
If `settings.json` contains a user without a `password` property then
nobody should be able to log in as that user using the built-in HTTP
basic authentication. This is true both with and without this change,
but before this change it wasn't immediately obvious that a malicious
user couldn't use an empty or null password to log in as such a user.
This commit adds an explicit nullish check and some unit tests to
ensure that an empty or null password will not work if the `password`
property is null or undefined.
This makes it possible to disable `contentEditable` for certain
elements in some circumstances (e.g., on links so that users can click
on them normally).
if animationState evaluates to -1 or 0, it would end up in a conditional that assign its value to itself. Since this is redundant, it is better to remove this conditional, to avoid an extra check
Rewrite the `callAll` and `aCallAll` functions to support all
reasonable hook behaviors and to report errors for unreasonable
behaviors (e.g., calling the callback twice).
Now a hook function like the following works as expected when invoked
by `aCallAll`:
```
exports.myHookFn = (hookName, context, cb) => {
cb('some value');
return;
};
```
If a hook function neither calls the callback nor returns a
(non-undefined) value then there's no way for the hook system to know
if/when the hook function has finished.
* don't include sendkeys in index.html as it's included in helper.init
mocha opts: add default timeout and replace ignoreLeaks with checkLeaks,
as the former is deprecated
* introduce helper.edit to write to a pad
* add test to check if helper.edit() supports line numbers
* helper tests: waitFor/waitForPromise seem to be a little bit faster sometimes
* tests: refactor chat.js
* tests: refactor timeslider_numeric_padID
* tests: refactor timeslider_labels
* tests: refactor timeslider_follow
* ensure followContents is enabled, although it should be by default
* timeslider_follow: increase number of revision for Edge
* make textLines() depend on linesDiv()
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* make linesDiv return standard Array
* use `contain` instead of `indexOf`
* more fixes from the review
* review fixes
* align waitFor and waitForPromise behaviour
* timeslider_follow: check if it's following to the correct lines
* lower expected waitFor/waitForPromise interval check
* disable responsivness and regression test in timeslider_follow
* timeslider_follow: fix Range detection
* more explicit test for linesDiv
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* Use jQuery to build the message HTML so that special characters in
the error message, URL, etc. are properly escaped. This helps
avoid XSS vulnerabilities.
* Use bold text for the error message to make it stand out.
* Add a line break between the error message and "in <url> at line
<line>" so that the error message stands out more.
* Use `<p>...</p>` instead of `</br>` to separate the parts of the
popup.
* Use CSS for spacing instead of `</br>`.
* Grammar fixes (add a missing comma, "at" instead of "in").
Teach Gritter to accept anything that jQuery's `.append()` method
accepts for the title and text of a popup message. This makes it
easier to safely build HTML messages with proper escaping of special
characters (to prevent XSS vulnerabilities).
* Avoid a false positive if a Promise that is expected to reject
doesn't reject.
* Use modern JavaScript language features: arrow functions,
`const`/`let` instead of `var`.
* Remove the tests that test Promise behavior.
* Add new test that checks that it returns a Promise.
There are a few problems with sleeping before checking the condition
for the first time:
* It slows down tests.
* The predicate is never checked if the interval duration is greater
than the timeout.
* 0 can't be used to test if the condition is currently true.
There is a minor disadvantage to sleeping before checking: It will
cause more tests to run without an asynchronous interruption, which
could theoretically mask some async bugs.
The `helper.waitFor()` function returns a jQuery Deferred object.
Deferred objects are supposed to have a `.fail()` method that is
chainable (it should return `this`). Before this change,
`helper.waitFor()` monkey-patched the `.fail()` method with a function
that returned `undefined`. Now the monkey-patched `.fail()` returns
the Deferred object.
Also modernize the code a bit.
The debug statement mostly printed the following useless message over
and over, causing Travis CI logs to become truncated:
[DEBUG] pluginfw - [ undefined ] returning
This will be a breaking change for some people.
We removed all internal password control logic. If this affects you, you have two options:
1. Use a plugin for authentication and use session based pad access (recommended).
1. Use a plugin for password setting.
The reasoning for removing this feature is to reduce the overall security footprint of Etherpad. It is unnecessary and cumbersome to keep this feature and with the thousands of available authentication methods available in the world our focus should be on supporting those and allowing more granual access based on their implementations (instead of half assed baking our own).
We could instead await the results of the hook, but then all callers
and their callers recursively would have to be converted to async, and
that's a huge change.
* Run node 10 with '--experimental_worker' flags
* Use dedicated function to retrieve node/npm program version
The goal of this commit is to ensure that any linux based node 10 deployments run with the experimental_worker flag. This flag is required for workers to "work" in node 10. This will not affect other versions of node. This resolves#4335 where Docker would fail due to being based on node 10.
This makes it easier to see the test results, and it hides some
scary-looking but intentional error messages.
This code will likely have to be updated if/when we change the logging
library (see issue #1922).
This currently isn't absolutely necessary because all current callers
of `userCanModify` already check for a read-only pad ID themselves.
However:
* This adds defense in depth.
* This makes it possible to simply replace the import handler's
`allowAnyoneToImport` check with a call to `userCanModify`.
This makes it possible to test various settings combinations and
examine internal state to confirm correct behavior. Also, the user
doesn't need to start an Etherpad server before running these tests.
There's no need to perform an authentication check in the socket.io
middleware because `PadMessageHandler.handleMessage` calls
`SecurityMananger.checkAccess` and that now performs authentication
and authorization checks.
This change also improves the user experience: Before, access denials
caused socket.io error events in the client, which `pad.js` mostly
ignores (the user doesn't see anything). Now a deny message is sent
back to the client, which causes `pad.js` to display an obvious
permission denied message.
This also fixes a minor bug: `settings.loadTest` is supposed to bypass
authentication and authorization checks, but they weren't bypassed
because `SecurityManager.checkAccess` did not check
`settings.loadTest`.
Rather than reinvent the wheel, use a well-tested library to parse and
write cookies. This should also help prevent XSS vulnerabilities
because the library handles special characters such as semicolon.
* Use the cookie functions from `pad_utils.js`.
* Delete unused methods, variables, and parameters.
* Simplify the logic.
* Use an ES6 class instead of a weird literal thingy.
* Use `const` instead of `var`.
Previously Etherpad would not pass the correct client IP address through and this caused the rate limiter to limit users behind reverse proxies. This change allows Etherpad to use a client IP passed from a reverse proxy.
Note to devs: This header can be spoofed and spoofing the header could be used in an attack. To mitigate additional *steps should be taken by Etherpad site admins IE doing rate limiting at proxy.* This only really applies to large scale deployments but it's worth noting.
Not every string was localized:
* `/admin/plugins` has some CSS magic to draw the tables of plugins
differently on narrow (mobile) screens, and the l10n library we
use does not support that particular magic. The strings that were
not localized are "Name", "Description", "Version", and "Time".
These strings are only stuck in English when the page is viewed on
a narrow screen; normal desktop users will see translated strings.
The CSS magic ought to be replaced with something more robust
(lots of nested `div`s); those remaining strings can be localized
whenever that happens.
* Strings from external sources such as plugin descriptions, error
messages, and `settings.json` comments are not localized.
Before this change, the authorize hook was invoked twice: once before
authentication and again after (if settings.requireAuthorization is
true). Now pre-authentication authorization is instead handled by a
new preAuthorize hook, and the authorize hook is only invoked after
the user has authenticated.
Rationale: Without this change it is too easy to write an
authorization plugin that is too permissive. Specifically:
* If the plugin does not check the path for /admin then a non-admin
user might be able to access /admin pages.
* If the plugin assumes that the user has already been authenticated
by the time the authorize function is called then unauthenticated
users might be able to gain access to restricted resources.
This change also avoids calling the plugin's authorize function twice
per access, which makes it easier for plugin authors to write an
authorization plugin that is easy to understand.
This change may break existing authorization plugins: After this
change, the authorize hook will no longer be able to authorize
non-admin access to /admin pages. This is intentional. Access to admin
pages should instead be controlled via the `is_admin` user setting,
which can be set in the config file or by an authentication plugin.
Also:
* Add tests for the authenticate and authorize hooks.
* Disable the authentication failure delay when testing.
This loses some of the granularity of the default HTTP basic auth
(unknown username vs. bad password), but there is considerable value
in having logging that is consistent no matter what authentication
plugins are installed.
The export request hook wasn't testing if the pad's id was from a read-only
pad before validating with the pad manager.
This includes an extra step that makes the read-only id verification and also
avoids setting the original pad's id as the file's name.
This PR introduces testing of plugins from the ether/ organization on Github. Each plugin is added into ``.travis``. Frontend plugins tests are run exclusive to core tests. Backend runs both core and plugins with core.
Including frontend core tests with plugin tests caused the session to overrun causing errors.
Commit 0bb8d73ba2 fixed the author ID
that is saved in the socket.io sessioninfo when the client sends a
`CLIENT_READY` with `reconnect` set to true, so it is now safe to undo
the workaround from PR #3868.
Fixes#4331.
Benefits:
* More functions are now async which makes it possible for future
changes to use await in those functions.
* This will help keep the server from drowning in too many messages
if we ever add acknowledgements or if WebSocket backpressure ever
becomes reality.
* This might make tests less flaky because changes triggered by a
message will complete before the Promise resolves.
Three of the four tests fail if `settings.allowAnyoneToImport` is
false. The fourth ("tries to import Plain Text to a pad that does not
exist") isn't particularly useful when `settings.allowAnyoneToImport`
is false: That test tests an import failure mode, and when
`settings.allowAnyoneToImport` is false the failure could be caused by
that instead of the expected cause.
Before, the author ID was only saved in the session info during the
initial CLIENT_READY, not when the client sent a CLIENT_READY due to a
reconnect. This caused the handling of subsequent messages to use an
undefined author ID.
This makes it possible for reverse proxies to transform 403 errors
into something like "upgrade to a premium account to access this
pad".
Also add some webaccess tests.
Move the handleMessageSecurity and handleMessage hooks after the call
to securityManager.checkAccess.
Benefits:
* A handleMessage plugin can safely assume the message will be
handled unless the plugin itself drops the message, so it doesn't
need to repeat the access checks done by the `handleMessage`
function.
* This paves the way for a future enhancement: pass the author ID to
the hooks.
Note: The handleMessageSecurity hook is broken in several ways:
* The hook result is ignored for `CLIENT_READY` and `SWITCH_TO_PAD`
messages because the `handleClientReady` function overwrites the
hook result. This causes the client to receive client vars with
`readonly` set to true, which causes the client to display an
immutable pad even though the pad is technically writable.
* The formatting toolbar buttons are removed for read-only pads
before the handleMessageSecurity hook even runs.
* It is awkwardly named: Without reading the documentation, how is
one supposed to know that "handle message security" actually means
"grant one-time write access to a read-only pad"?
* It is called for every message even though calls after a
`CLIENT_READY` or `SWITCH_TO_PAD` are mostly pointless.
* Why would anyone want to grant write access when the user visits a
read-only pad URL? The user should just visit the writable pad URL
instead.
* Why would anyone want to grant write access that only lasts for a
single socket.io connection?
* There are better ways to temporarily grant write access (e.g., the
authorize hook).
* This hook is inviting bugs because it breaks a core assumption
about `/p/r.*` URLs.
I think the hook should be deprecated and eventually removed.
<!-- IMPORTANT: Please disable plugins prior to posting a bug report. If you have a problem with a plugin please post on the plugin repository. Thanks! -->
**Describe the bug**
A clear and concise description of what the bug is.
@ -23,6 +25,13 @@ A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Server (please complete the following information):**
- Etherpad version:
- OS: [e.g., Ubuntu 20.04]
- Node.js version (`node --version`):
- npm version (`npm --version`):
- Is the server free of plugins:
**Desktop (please complete the following information):**
# This Action will scan dependency manifest files that change as part of a Pull Reqest, surfacing known-vulnerable versions of the packages declared or updated in the PR. Once installed, if the workflow run is marked as required, PRs introducing known-vulnerable packages will be blocked from merging.
# Public documentation: https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-dependency-review#dependency-review-enforcement
* Log4js has been updated to the latest version. As it involved a bump of 6 major version.
A lot has changed since then. Most notably the console appender has been deprecated. You can find out more about it [here](https://github.com/log4js-node/log4js-node)
### Notable enhancements and fixes
* Fix for MySQL: The logger calls were incorrectly configured leading to a crash when e.g. somebody uses a different encoding than standard MySQL encoding.
# 1.9.3
### Compability changes
* express-rate-limit has been bumped to 7.0.0: This involves the breaking change that "max: 0"
in the importExportRateLimiting is set to always trigger. So set it to your desired value.
If you haven't changed that value in the settings.json you are all set.
### Notable enhancements and fixes
* Bugfixes
* Fix etherpad crashing with mongodb database
* Enhancements
* Add surrealdb database support. You can find out more about this database [here](https://surrealdb.com).
* Make sqlite faster: The sqlite library has been switched to better-sqlite3. This should lead to better performance.
# 1.9.2
### Notable enhancements and fixes
* Security
* Enable session key rotation: This setting can be enabled in the settings.json. It changes the signing key for the cookie authentication in a fixed interval.
* Bugfixes
* Fix appendRevision when creating a new pad via the API without a text.
* Enhancements
* Bump JQuery to version 3.7
* Update elasticsearch connector to version 8
### Compatibility changes
* No compability changes as JQuery maintains excellent backwards compatibility.
#### For plugin authors
* Please update to JQuery 3.7. There is an excellent deprecation guide over [here](https://api.jquery.com/category/deprecated/). Version 3.1 to 3.7 are relevant for the upgrade.
# 1.9.1
### Notable enhancements and fixes
* Security
* Limit requested revisions in timeslider and export to head revision. (affects v1.9.0)
* Bugfixes
* revisions in `CHANGESET_REQ` (timeslider) and export (txt, html, custom)
are now checked to be numbers.
* bump sql for audit fix
* Enhancements
* Add keybinding meta-backspace to delete to beginning of line
* Fix automatic Windows build via GitHub Actions
* Enable docs to be build cross platform thanks to asciidoctor
### Compatibility changes
* tests: drop windows 7 test coverage & use chrome latest for admin tests
* Require Node 16 for Etherpad and target Node 20 for testing
# 1.9.0
### Notable enhancements and fixes
* Windows build:
* The bundled `node.exe` was upgraded from v12 to v16.
* The bundled `node.exe` is now a 64-bit executable. If you need the 32-bit
version you must download and install Node.js yourself.
* Improvements to login session management:
* `express_sid` cookies and `sessionstorage:*` database records are no longer
created unless `requireAuthentication` is `true` (or a plugin causes them to
be created).
* Login sessions now have a finite lifetime by default (10 days after
leaving).
* `sessionstorage:*` database records are automatically deleted when the login
session expires (with some exceptions that will be fixed in the future).
* Requests for static content (e.g., `/robots.txt`) and special pages (e.g.,
the HTTP API, `/stats`) no longer create login session state.
* The secret used to sign the `express_sid` cookie is now automatically
regenerated every day (called *key rotation*) by default. If key rotation is
enabled, the now-deprecated `SESSIONKEY.txt` file can be safely deleted
after Etherpad starts up (its content is read and saved to the database and
used to validate signatures from old cookies until they expire).
* The following settings from `settings.json` are now applied as expected (they
were unintentionally ignored before):
* `padOptions.lang`
* `padOptions.showChat`
* `padOptions.userColor`
* `padOptions.userName`
* HTTP API:
* Fixed the return value of `getText` when called with a specific revision.
* Fixed a potential attribute pool corruption bug with
`copyPadWithoutHistory`.
* Mappings created by `createGroupIfNotExistsFor` are now removed from the
database when the group is deleted.
* Fixed race conditions in the `setText`, `appendText`, and `restoreRevision`
functions.
* Added an optional `authorId` parameter to `appendText`,
number of the issue that is being fixed, in the form: Fixes #someIssueNumber
```
* if the PR is a **bug fix**:
* the first commit in the series must be a test that shows the failure
* subsequent commits will fix the bug and make the test pass
* the final commit message should include the text `Fixes: #xxx` to link it to its bug report
* The commit that fixes the bug should **include a regression test** that
would fail if the bug fix was reverted. Adding the regression test in the
same commit as the bug fix makes it easier for a reviewer to verify that the
test is appropriate for the bug fix.
* If there is a bug report, **the pull request description should include the
text "`Fixes #xxx`"** so that the bug report is auto-closed when the PR is
merged. It is less useful to say the same thing in a commit message because
GitHub will spam the bug report every time the commit is rebased, and
because a bug number alone becomes meaningless in forks. (A full URL would
be better, but ideally each commit is readable on its own without the need
to examine an external reference to understand motivation or context.)
* think about stability: code has to be backwards compatible as much as possible. Always **assume your code will be run with an older version of the DB/config file**
* if you want to remove a feature, **deprecate it instead**:
* write an issue with your deprecation plan
@ -116,6 +124,7 @@ You can build the docs e.g. produce html, using `make docs`. At some point in th
Front-end tests are found in the `tests/frontend/` folder in the repository. Run them by pointing your browser to `<yourdomainhere>/tests/frontend`.
Back-end tests can be run from the `src` directory, via `npm test`.
You can use `npm test -- --inspect-brk` and navigate to `edge://inspect` or `chrome://inspect` to debug the tests.
## Things you can help with
Etherpad is much more than software. So if you aren't a developer then worry not, there is still a LOT you can do! A big part of what we do is community engagement. You can help in the following ways
# Etherpad: A real-time collaborative editor for the web
# About
Etherpad is a real-time collaborative editor scalable to thousands of simultaneous real time users. It provides full data export capabilities, and runs on _your_ server, under _your_ control.
## About
**[Try it out](https://video.etherpad.com)**
Etherpad is a real-time collaborative editor [scalable to thousands of
simultaneous real time users](http://scale.etherpad.org/). It provides [full
capabilities, and runs on _your_ server, under _your_ control.
# Installation
## Try it out
## Requirements
- `nodejs` >= **10.13.0**.
Wikimedia provide a [public Etherpad instance for you to Try Etherpad out.](https://etherpad.wikimedia.org) or [use another public Etherpad instance to see other features](https://github.com/ether/etherpad-lite/wiki/Sites-That-Run-Etherpad#sites-that-run-etherpad)
2. With a "Run as administrator" command prompt execute
`src\bin\installOnWindows.bat`
Update to the latest version with `git pull origin`, then run `bin\installOnWindows.bat`, again.
Now, run `start.bat` and open http://localhost:9001 in your browser.
If cloning to a subdirectory within another project, you may need to do the following:
Update to the latest version with `git pull origin`, then run
`src\bin\installOnWindows.bat`, again.
1. Start the server manually (e.g. `node/node_modules/ep_etherpad-lite/node/server.js`)
2. Edit the db `filename` in `settings.json` to the relative directory with the file (e.g. `application/lib/etherpad-lite/var/dirty.db`)
3. Add auto-generated files to the main project `.gitignore`
If cloning to a subdirectory within another project, you may need to do the
following:
## Docker container
1. Start the server manually (e.g. `node src/node/server.js`)
2. Edit the db `filename` in `settings.json` to the relative directory with
the file (e.g. `application/lib/etherpad-lite/var/dirty.db`)
3. Add auto-generated files to the main project `.gitignore`
Find [here](doc/docker.md) information on running Etherpad in a container.
### Docker container
# Next Steps
Find [here](doc/docker.adoc) information on running Etherpad in a container.
## Tweak the settings
You can modify the settings in `settings.json`.
If you need to handle multiple settings files, you can pass the path to a settings file to `bin/run.sh` using the `-s|--settings` option: this allows you to run multiple Etherpad instances from the same installation.
Similarly, `--credentials` can be used to give a settings override file, `--apikey` to give a different APIKEY.txt file and `--sessionkey` to give a non-default SESSIONKEY.txt.
**Each configuration parameter can also be set via an environment variable**, using the syntax `"${ENV_VAR}"` or `"${ENV_VAR:default_value}"`. For details, refer to `settings.json.template`.
Once you have access to your `/admin` section settings can be modified through the web browser.
## Plugins
If you are planning to use Etherpad in a production environment, you should use a dedicated database such as `mysql`, since the `dirtyDB` database driver is only for testing and/or development purposes.
## Secure your installation
If you have enabled authentication in `users` section in `settings.json`, it is a good security practice to **store hashes instead of plain text passwords** in that file. This is _especially_ advised if you are running a production installation.
Please install [ep_hash_auth plugin](https://www.npmjs.com/package/ep_hash_auth) and configure it.
If you prefer, `ep_hash_auth` also gives you the option of storing the users in a custom directory in the file system, without having to edit `settings.json` and restart Etherpad each time.
Etherpad is very customizable through plugins. Instructions for installing themes and plugins can be found in [the plugin wiki article](https://github.com/ether/etherpad-lite/wiki/Available-Plugins).
### Available Plugins
## Getting the full features
Run the following command in your Etherpad folder to get all of the features visible in the demo gif:
The [wiki](https://github.com/ether/etherpad-lite/wiki) is your one-stop resource for Tutorials and How-to's.
The [wiki](https://github.com/ether/etherpad-lite/wiki) is your one-stop
resource for Tutorials and How-to's.
Documentation can be found in `doc/`.
# Development
## Development
## Things you should know
You can debug Etherpad using `bin/debugRun.sh`.
### Things you should know
You can run Etherpad quickly launching `bin/fastRun.sh`. It's convenient for developers and advanced users. Be aware that it will skip the dependencies update, so remember to run `bin/installDeps.sh` after installing a new dependency or upgrading version.
You can debug Etherpad using `src/bin/debugRun.sh`.
If you want to find out how Etherpad's `Easysync` works (the library that makes it really realtime), start with this [PDF](https://github.com/ether/etherpad-lite/raw/master/doc/easysync/easysync-full-description.pdf) (complex, but worth reading).
You can run Etherpad quickly launching `src/bin/fastRun.sh`. It's convenient for
developers and advanced users. Be aware that it will skip the dependencies
update, so remember to run `src/bin/installDeps.sh` after installing a new
Etherpad is designed to be easily embeddable and provides a [HTTP API](https://github.com/ether/etherpad-lite/wiki/HTTP-API)
that allows your web application to manage pads, users and groups. It is recommended to use the [available client implementations](https://github.com/ether/etherpad-lite/wiki/HTTP-API-client-libraries) in order to interact with this API.
### HTTP API
OpenAPI (previously swagger) definitions for the API are exposed under `/api/openapi.json`.
Etherpad is designed to be easily embeddable and provides a [HTTP
API](https://github.com/ether/etherpad-lite/wiki/HTTP-API) that allows your web
application to manage pads, users and groups. It is recommended to use the
There is a [jQuery plugin](https://github.com/ether/etherpad-lite-jquery-plugin) that helps you to embed Pads into your website.
OpenAPI (previously swagger) definitions for the API are exposed under
`/api/openapi.json`.
# Plugin Framework
Etherpad offers a plugin framework, allowing you to easily add your own features. By default your Etherpad is extremely light-weight and it's up to you to customize your experience. Once you have Etherpad installed you should [visit the plugin page](https://static.etherpad.org/) and take control.
### jQuery plugin
# Translations / Localizations (i18n / l10n)
Etherpad comes with translations into all languages thanks to the team at [TranslateWiki](https://translatewiki.net/).
There is a [jQuery plugin](https://github.com/ether/etherpad-lite-jquery-plugin)
that helps you to embed Pads into your website.
If you require translations in [plugins](https://static.etherpad.org/) please send pull request to each plugin individually.
### Plugin Framework
Etherpad offers a plugin framework, allowing you to easily add your own
features. By default your Etherpad is extremely light-weight and it's up to you
to customize your experience. Once you have Etherpad installed you should [visit
the plugin page](https://static.etherpad.org/) and take control.
### Translations / Localizations (i18n / l10n)
Etherpad comes with translations into all languages thanks to the team at
[TranslateWiki](https://translatewiki.net/).
If you require translations in [plugins](https://static.etherpad.org/) please
send pull request to each plugin individually.
## FAQ
# FAQ
Visit the **[FAQ](https://github.com/ether/etherpad-lite/wiki/FAQ)**.
# License
## Get in touch
The official channel for contacting the development team is via the [GitHub
console.warn(".travis.yml file not found, please create. .travis.yml is used for automatically CI testing Etherpad. It is useful to know if your plugin breaks another feature for example.")
// TODO: Make it check version of the .travis file to see if it needs an update.
console.warn("Translations not found, please create. Translation files help with Etherpad accessibility.");
}
if(files.indexOf(".ep_initialized")!==-1){
console.warn(".ep_initialized found, please remove. .ep_initialized should never be commited to git and should only exist once the plugin has been executed one time.")
Changesets do not include any attribute key–value pairs. Instead, they use
numeric identifiers that reference attributes kept in an https://github.com/ether/etherpad-lite/blob/develop/src/static/js/AttributePool.js[attribute pool].
This attribute interning reduces the transmission overhead of attributes that
are used many times.
There is one attribute pool per pad, and it includes every current and
historical attribute used in the pad.
=== Further Reading
Detailed information about the changesets & Easysync protocol:
This is a Changeset. It's just a string and it's very difficult to read in this form. But the Changeset Library gives us some tools to read it.
A changeset describes the diff between two revisions of the document. The Browser sends changesets to the server and the server sends them to the clients to update them. These Changesets also get saved into the history of a pad. This allows us to go back to every revision from the past.
## Changeset.unpack(changeset)
* `changeset` {String}
This function returns an object representation of the changeset, similar to this:
* `oldLen` {Number} the original length of the document.
* `newLen` {Number} the length of the document after the changeset is applied.
* `ops` {String} the actual changes, introduced by this changeset.
* `charBank` {String} All characters that are added by this changeset.
## Changeset.opIterator(ops)
* `ops` {String} The operators, returned by `Changeset.unpack()`
Returns an operator iterator. This iterator allows us to iterate over all operators that are in the changeset.
You can iterate with an opIterator using its `next()` and `hasNext()` methods. Next returns the `next()` operator object and `hasNext()` indicates, whether there are any operators left.
## The Operator object
There are 3 types of operators: `+`,`-` and `=`. These operators describe different changes to the document, beginning with the first character of the document. A `=` operator doesn't change the text, but it may add or remove text attributes. A `-` operator removes text. And a `+` Operator adds text and optionally adds some attributes to it.
* `opcode` {String} the operator type
* `chars` {Number} the length of the text changed by this operator.
* `lines` {Number} the number of lines changed by this operator.
* `attribs` {attribs} attributes set on this text.
### Example
```
{ opcode: '+',
chars: 1,
lines: 1,
attribs: '*0' }
```
## APool
```
> var AttributePoolFactory = require("./utils/AttributePoolFactory");
> var apool = AttributePoolFactory.createAttributePool();
> console.log(apool)
{ numToAttrib: {},
attribToNum: {},
nextNum: 0,
putAttrib: [Function],
getAttrib: [Function],
getAttribKey: [Function],
getAttribValue: [Function],
eachAttrib: [Function],
toJsonable: [Function],
fromJsonable: [Function] }
```
This creates an empty apool. An apool saves which attributes were used during the history of a pad. There is one apool for each pad. It only saves the attributes that were really used, it doesn't save unused attributes. Let's fill this apool with some values
We used the fromJsonable function to fill the empty apool with values. the fromJsonable and toJsonable functions are used to serialize and deserialize an apool. You can see that it stores the relation between numbers and attributes. So for example the attribute 1 is the attribute bold and vise versa. An attribute is always a key value pair. For stuff like bold and italic it's just 'italic':'true'. For authors it's author:$AUTHORID. So a character can be bold and italic. But it can't belong to multiple authors
```
> apool.getAttrib(1)
[ 'bold', 'true' ]
```
Simple example of how to get the key value pair for the attribute 1
## AText
```
> var atext = {"text":"bold text\nitalic text\nnormal text\n\n","attribs":"*0*1+9*0|1+1*0*1*2+b|1+1*0+b|2+2"};
This is an atext. An atext has two parts: text and attribs. The text is just the text of the pad as a string. We will look closer at the attribs at the next steps
```
> var opiterator = Changeset.opIterator(atext.attribs)
> console.log(opiterator)
{ next: [Function: next],
hasNext: [Function: hasNext],
lastIndex: [Function: lastIndex] }
> opiterator.next()
{ opcode: '+',
chars: 9,
lines: 0,
attribs: '*0*1' }
> opiterator.next()
{ opcode: '+',
chars: 1,
lines: 1,
attribs: '*0' }
> opiterator.next()
{ opcode: '+',
chars: 11,
lines: 0,
attribs: '*0*1*2' }
> opiterator.next()
{ opcode: '+',
chars: 1,
lines: 1,
attribs: '' }
> opiterator.next()
{ opcode: '+',
chars: 11,
lines: 0,
attribs: '*0' }
> opiterator.next()
{ opcode: '+',
chars: 2,
lines: 2,
attribs: '' }
```
The attribs are again a bunch of operators like .ops in the changeset was. But these operators are only + operators. They describe which part of the text has which attributes
For more information see /doc/easysync/easysync-notes.txt in the source.
You can easily embed your etherpad-lite into any webpage by using iframes. You can configure the embedded pad using embed parameters.
Example:
Cut and paste the following code into any webpage to embed a pad. The parameters below will hide the chat and the line numbers.
Cut and paste the following code into any webpage to embed a pad. The parameters below will hide the chat and the line numbers and will auto-focus on Line 4.
Most of these hooks are called during or in order to set up the formatting process.
== Client-side hooks
## documentReady
Most of these hooks are called during or in order to set up the formatting
process.
=== documentReady
Called from: src/templates/pad.html
Things in context:
@ -10,7 +12,8 @@ nothing
This hook proxies the functionality of jQuery's `$(document).ready` event.
## aceDomLinePreProcessLineAttributes
=== aceDomLinePreProcessLineAttributes
Called from: src/static/js/domline.js
Things in context:
@ -18,15 +21,21 @@ Things in context:
1. domline - The current DOM line being processed
2. cls - The class of the current block element (useful for styling)
This hook is called for elements in the DOM that have the "lineMarkerAttribute" set. You can add elements into this category with the aceRegisterBlockElements hook above. This hook is run BEFORE the numbered and ordered lists logic is applied.
This hook is called for elements in the DOM that have the "lineMarkerAttribute"
set. You can add elements into this category with the aceRegisterBlockElements
hook above. This hook is run BEFORE the numbered and ordered lists logic is
applied.
The return value of this hook should have the following structure:
The preHtml and postHtml values will be added to the HTML display of the element, and if processedMarker is true, the engine won't try to process it any more.
The preHtml and postHtml values will be added to the HTML display of the
element, and if processedMarker is true, the engine won't try to process it any
more.
=== aceDomLineProcessLineAttributes
## aceDomLineProcessLineAttributes
Called from: src/static/js/domline.js
Things in context:
@ -34,15 +43,21 @@ Things in context:
1. domline - The current DOM line being processed
2. cls - The class of the current block element (useful for styling)
This hook is called for elements in the DOM that have the "lineMarkerAttribute" set. You can add elements into this category with the aceRegisterBlockElements hook above. This hook is run AFTER the ordered and numbered lists logic is applied.
This hook is called for elements in the DOM that have the "lineMarkerAttribute"
set. You can add elements into this category with the aceRegisterBlockElements
hook above. This hook is run AFTER the ordered and numbered lists logic is
applied.
The return value of this hook should have the following structure:
The preHtml and postHtml values will be added to the HTML display of the element, and if processedMarker is true, the engine won't try to process it any more.
The preHtml and postHtml values will be added to the HTML display of the
element, and if processedMarker is true, the engine won't try to process it any
more.
=== aceCreateDomLine
## aceCreateDomLine
Called from: src/static/js/domline.js
Things in context:
@ -50,79 +65,106 @@ Things in context:
1. domline - the current DOM line being processed
2. cls - The class of the current element (useful for styling)
This hook is called for any line being processed by the formatting engine, unless the aceDomLineProcessLineAttributes hook from above returned true, in which case this hook is skipped.
This hook is called for any line being processed by the formatting engine,
unless the aceDomLineProcessLineAttributes hook from above returned true, in
which case this hook is skipped.
The return value of this hook should have the following structure:
extraOpenTags and extraCloseTags will be added before and after the element in question, and cls will be the new class of the element going forward.
extraOpenTags and extraCloseTags will be added before and after the element in
question, and cls will be the new class of the element going forward.
=== acePostWriteDomLineHTML
## acePostWriteDomLineHTML
Called from: src/static/js/domline.js
Things in context:
1. node - the DOM node that just got written to the page
This hook is for right after a node has been fully formatted and written to the page.
This hook is for right after a node has been fully formatted and written to the
page.
=== aceAttribsToClasses
## aceAttribsToClasses
Called from: src/static/js/linestylefilter.js
Things in context:
1. linestylefilter - the JavaScript object that's currently processing the ace attributes
1. linestylefilter - the JavaScript object that's currently processing the ace
attributes
2. key - the current attribute being processed
3. value - the value of the attribute being processed
This hook is called during the attribute processing procedure, and should be used to translate key, value pairs into valid HTML classes that can be inserted into the DOM.
This hook is called during the attribute processing procedure, and should be
used to translate key, value pairs into valid HTML classes that can be inserted
into the DOM.
The return value for this function should be a list of classes, which will then be parsed into a valid class string.
The return value for this function should be a list of classes, which will then
be parsed into a valid class string.
=== aceAttribClasses
## aceAttribClasses
Called from: src/static/js/linestylefilter.js
Things in context:
1. Attributes - Object of Attributes
This hook is called when attributes are investigated on a line. It is useful if you want to add another attribute type or property type to a pad.
This hook is called when attributes are investigated on a line. It is useful if
you want to add another attribute type or property type to a pad.
1. linestylefilter - the JavaScript object that's currently processing the ace attributes
1. linestylefilter - the JavaScript object that's currently processing the ace
attributes
2. browser - an object indicating which browser is accessing the page
This hook is called to apply custom regular expression filters to a set of styles. The one example available is the ep_linkify plugin, which adds internal links. They use it to find the telltale `[[ ]]` syntax that signifies internal links, and finding that syntax, they add in the internalHref attribute to be later used by the aceCreateDomLine hook (documented above).
This hook is called to apply custom regular expression filters to a set of
styles. The one example available is the ep_linkify plugin, which adds internal
links. They use it to find the telltale `[[ ]]` syntax that signifies internal
links, and finding that syntax, they add in the internalHref attribute to be
later used by the aceCreateDomLine hook (documented above).
=== aceEditorCSS
## aceEditorCSS
Called from: src/static/js/ace.js
Things in context: None
This hook is provided to allow custom CSS files to be loaded. The return value should be an array of resource urls or paths relative to the plugins directory.
This hook is provided to allow custom CSS files to be loaded. The return value
should be an array of resource urls or paths relative to the plugins directory.
=== aceInitInnerdocbodyHead
## aceInitInnerdocbodyHead
Called from: src/static/js/ace.js
Things in context:
1. iframeHTML - the HTML of the editor iframe up to this point, in array format
This hook is called during the creation of the editor HTML. The array should have lines of HTML added to it, giving the plugin author a chance to add in meta, script, link, and other tags that go into the `<head>` element of the editor HTML document.
This hook is called during the creation of the editor HTML. The array should
have lines of HTML added to it, giving the plugin author a chance to add in
meta, script, link, and other tags that go into the `<head>` element of the
editor HTML document.
=== aceEditEvent
## aceEditEvent
Called from: src/static/js/ace2_inner.js
Things in context:
@ -130,51 +172,74 @@ Things in context:
1. callstack - a bunch of information about the current action
2. editorInfo - information about the user who is making the change
3. rep - information about where the change is being made
4. documentAttributeManager - information about attributes in the document (this is a mystery to me)
4. documentAttributeManager - information about attributes in the document (this
is a mystery to me)
This hook is made available to edit the edit events that might occur when changes are made. Currently you can change the editor information, some of the meanings of the edit, and so on. You can also make internal changes (internal to your plugin) that use the information provided by the edit event.
This hook is made available to edit the edit events that might occur when
changes are made. Currently you can change the editor information, some of the
meanings of the edit, and so on. You can also make internal changes (internal to
your plugin) that use the information provided by the edit event.
=== aceRegisterNonScrollableEditEvents
## aceRegisterNonScrollableEditEvents
Called from: src/static/js/ace2_inner.js
Things in context: None
When aceEditEvent (documented above) finishes processing the event, it scrolls the viewport to make caret visible to the user, but if you don't want that behavior to happen you can use this hook to register which edit events should not scroll viewport. The return value of this hook should be a list of event names.
When aceEditEvent (documented above) finishes processing the event, it scrolls
the viewport to make caret visible to the user, but if you don't want that
behavior to happen you can use this hook to register which edit events should
not scroll viewport. The return value of this hook should be a list of event
The return value of this hook will add elements into the "lineMarkerAttribute" category, making the aceDomLineProcessLineAttributes hook (documented below) call for those elements.
The return value of this hook will add elements into the "lineMarkerAttribute"
category, making the aceDomLineProcessLineAttributes hook (documented below)
call for those elements.
=== aceInitialized
## aceInitialized
Called from: src/static/js/ace2_inner.js
Things in context:
1. editorInfo - information about the user who will be making changes through the interface, and a way to insert functions into the main ace object (see ep_headings)
1. editorInfo - information about the user who will be making changes through
the interface, and a way to insert functions into the main ace object (see
ep_headings)
2. rep - information about where the user's cursor is
3. documentAttributeManager - some kind of magic
This hook is for inserting further information into the ace engine, for later use in formatting hooks.
This hook is for inserting further information into the ace engine, for later
use in formatting hooks.
=== postAceInit
## postAceInit
Called from: src/static/js/pad.js
Things in context:
1. ace - the ace object that is applied to this editor.
2. pad - the pad object of the current pad.
2. clientVars - Object containing client-side configuration such as author ID
and plugin settings. Your plugin can manipulate this object via the
`clientVars` server-side hook.
3. pad - the pad object of the current pad.
=== postToolbarInit
## postToolbarInit
Called from: src/static/js/pad_editbar.js
Things in context:
@ -186,48 +251,88 @@ Can be used to register custom actions to the toolbar.
There doesn't appear to be any example available of this particular hook being used, but it gets fired after the timeslider is all set up.
There doesn't appear to be any example available of this particular hook being
used, but it gets fired after the timeslider is all set up.
=== goToRevisionEvent
## goToRevisionEvent
Called from: src/static/js/broadcast.js
Things in context:
1. rev - The newRevision
This hook gets fired both on timeslider load (as timeslider shows a new revision) and when the new revision is showed to a user.
There doesn't appear to be any example available of this particular hook being used.
This hook gets fired both on timeslider load (as timeslider shows a new
revision) and when the new revision is showed to a user. There doesn't appear to
be any example available of this particular hook being used.
=== userJoinOrUpdate
## userJoinOrUpdate
Called from: src/static/js/pad_userlist.js
Things in context:
1. info - the user information
This hook is called on the client side whenever a user joins or changes. This can be used to create notifications or an alternate user list.
This hook is called on the client side whenever a user joins or changes. This
can be used to create notifications or an alternate user list.
## chatNewMessage
Called from: src/static/js/chat.js
=== chatNewMessage
Things in context:
Called from: `src/static/js/chat.js`
1. authorName - The user that wrote this message
2. author - The authorID of the user that wrote the message
3. text - the message text
4. sticky (boolean) - if you want the gritter notification bubble to fade out on its own or just sit there
5. timestamp - the timestamp of the chat message
6. timeStr - the timestamp as a formatted string
7. duration - for how long in milliseconds should the gritter notification appear (0 to disable)
This hook runs on the client side whenever a chat message is received from the
server. It can be used to create different notifications for chat messages. Hook
functions can modify the `author`, `authorName`, `duration`, `rendered`,
`sticky`, `text`, and `timeStr` context properties to change how the message is
processed. The `text` and `timeStr` properties may contain HTML and come
pre-sanitized; plugins should be careful to sanitize any added user input to
avoid introducing an XSS vulnerability.
This hook is called on the client side whenever a chat message is received from the server. It can be used to create different notifications for chat messages.
Context properties:
* `authorName`: The display name of the user that wrote the message.
* `author`: The author ID of the user that wrote the message.
* `text`: Sanitized message HTML, with URLs wrapped like `<a
href="url">url</a>`. (Note that `message.text` is not sanitized or processed
in any way.)
* `message`: The raw message object as received from the server, except with
time correction and a default `authorId` property if missing. Plugins must not
modify this object. Warning: Unlike `text`, `message.text` is not
pre-sanitized or processed in any way.
* `rendered` - Used to override the default message rendering. Initially set to
`null`. If the hook function sets this to a DOM element object or a jQuery
object, then that object will be used as the rendered message UI. Otherwise,
if this is set to `null`, then Etherpad will render a default UI for the
message using the other context properties.
* `sticky` (boolean): Whether the gritter notification should fade out on its
own or just sit there until manually closed.
* `timestamp`: When the chat message was sent (milliseconds since epoch),
corrected using the difference between the local clock and the server's clock.
* `timeStr`: The message timestamp as a formatted string.
* `duration`: How long (in milliseconds) to display the gritter notification (0
to disable).
=== chatSendMessage
Called from: `src/static/js/chat.js`
This hook runs on the client side whenever the user sends a new chat message.
Plugins can mutate the message object to change the message text or add metadata
to control how the message will be rendered by the `chatNewMessage` hook.
Context properties:
* `message`: The message object that will be sent to the Etherpad server.
=== collectContentPre
## collectContentPre
Called from: src/static/js/contentcollector.js
Things in context:
@ -238,16 +343,20 @@ Things in context:
4. styl - the style applied to the node (probably CSS) -- Note the typo
5. cls - the HTML class string of the node
This hook is called before the content of a node is collected by the usual methods. The cc object can be used to do a bunch of things that modify the content of the pad. See, for example, the heading1 plugin for etherpad original.
This hook is called before the content of a node is collected by the usual
methods. The cc object can be used to do a bunch of things that modify the
content of the pad. See, for example, the heading1 plugin for etherpad original.
E.g. if you need to apply an attribute to newly inserted characters,
call cc.doAttrib(state, "attributeName") which results in an attribute attributeName=true.
E.g. if you need to apply an attribute to newly inserted characters, call
cc.doAttrib(state, "attributeName") which results in an attribute
attributeName=true.
If you want to specify also a value, call cc.doAttrib(state, "attributeName::value")
which results in an attribute attributeName=value.
If you want to specify also a value, call cc.doAttrib(state,
"attributeName::value") which results in an attribute attributeName=value.
## collectContentImage
=== collectContentImage
Called from: src/static/js/contentcollector.js
Things in context:
@ -259,18 +368,22 @@ Things in context:
5. cls - the HTML class string of the node
6. node - the node being modified
This hook is called before the content of an image node is collected by the usual methods. The cc object can be used to do a bunch of things that modify the content of the pad.
This hook is called before the content of an image node is collected by the
usual methods. The cc object can be used to do a bunch of things that modify the
4. style - the style applied to the node (probably CSS)
5. cls - the HTML class string of the node
This hook is called after the content of a node is collected by the usual methods. The cc object can be used to do a bunch of things that modify the content of the pad. See, for example, the heading1 plugin for etherpad original.
This hook is called after the content of a node is collected by the usual
methods. The cc object can be used to do a bunch of things that modify the
content of the pad. See, for example, the heading1 plugin for etherpad original.
=== handleClientMessage_`name`
## handleClientMessage_`name`
Called from: `src/static/js/collab_client.js`
Things in context:
1. payload - the data that got sent with the message (use it for custom message content)
1. payload - the data that got sent with the message (use it for custom message
content)
This hook gets called every time the client receives a message of type `name`. This can most notably be used with the new HTTP API call, "sendClientsMessage", which sends a custom message type to all clients connected to a pad. You can also use this to handle existing types.
This hook gets called every time the client receives a message of type `name`.
This can most notably be used with the new HTTP API call, "sendClientsMessage",
which sends a custom message type to all clients connected to a pad. You can
also use this to handle existing types.
`collab_client.js` has a pretty extensive list of message types, if you want to take a look.
`collab_client.js` has a pretty extensive list of message types, if you want to
2. state - the current state of the change being made
3. tname - the tag name of this node currently being processed
This hook is provided to allow whether the br tag should induce a new magic domline or not.
The return value should be either true(break the line) or false.
This hook is provided to allow whether the br tag should induce a new magic
domline or not. The return value should be either true(break the line) or false.
=== disableAuthorColorsForThisLine
##disableAuthorColorsForThisLine
Called from: src/static/js/linestylefilter.js
Things in context:
1. linestylefilter - the JavaScript object that's currently processing the ace attributes
1. linestylefilter - the JavaScript object that's currently processing the ace
attributes
2. text - the line text
3. class - line class
This hook is provided to allow whether a given line should be deliniated with multiple authors.
Multiple authors in one line cause the creation of magic span lines. This might not suit you and
now you can disable it and handle your own deliniation.
The return value should be either true(disable) or false.
This hook is provided to allow whether a given line should be deliniated with
multiple authors. Multiple authors in one line cause the creation of magic span
lines. This might not suit you and now you can disable it and handle your own
deliniation. The return value should be either true(disable) or false.
=== aceSetAuthorStyle
## aceSetAuthorStyle
Called from: src/static/js/ace2_inner.js
Things in context:
@ -374,10 +516,12 @@ Things in context:
5. author - author info
6. authorSelector - css selector for author span in inner ace
This hook is provided to allow author highlight style to be modified.
Registered hooks should return 1 if the plugin handles highlighting. If no plugin returns 1, the core will use the default background-based highlighting.
This hook is provided to allow author highlight style to be modified. Registered
hooks should return 1 if the plugin handles highlighting. If no plugin returns
1, the core will use the default background-based highlighting.
Use this hook to receive the global settings in your plugin.
## shutdown
Called from: src/node/server.js
Things in context: None
This hook runs before shutdown. Use it to stop timers, close sockets and files,
flush buffers, etc. The database is not available while this hook is running.
The shutdown function must not block for long because there is a short timeout
before the process is forcibly terminated.
The shutdown function must return a Promise, which must resolve to `undefined`.
Returning `callback(value)` will return a Promise that is resolved to `value`.
Example:
```
// using an async function
exports.shutdown = async (hookName, context) => {
await flushBuffers();
};
```
## pluginUninstall
Called from: src/static/js/pluginfw/installer.js
Things in context:
1. plugin_name - self-explanatory
If this hook returns an error, the callback to the uninstall function gets an error as well. This mostly seems useful for handling additional features added in based on the installation of other plugins, which is pretty cool!
## pluginInstall
Called from: src/static/js/pluginfw/installer.js
Things in context:
1. plugin_name - self-explanatory
If this hook returns an error, the callback to the install function gets an error, too. This seems useful for adding in features when a particular plugin is installed.
## init_`<pluginname>`
Called from: src/static/js/pluginfw/plugins.js
Things in context: None
This function is called after a specific plugin is initialized. This would probably be more useful than the previous two functions if you only wanted to add in features to one specific plugin.
## expressConfigure
Called from: src/node/hooks/express.js
Things in context:
1. app - the main application object
This is a helpful hook for changing the behavior and configuration of the application. It's called right after the application gets configured.
## expressCreateServer
Called from: src/node/hooks/express.js
Things in context:
1. app - the main express application object (helpful for adding new paths and such)
2. server - the http server object
This hook gets called after the application object has been created, but before it starts listening. This is similar to the expressConfigure hook, but it's not guaranteed that the application object will have all relevant configuration variables.
## eejsBlock_`<name>`
Called from: src/node/eejs/index.js
Things in context:
1. content - the content of the block
This hook gets called upon the rendering of an ejs template block. For any specific kind of block, you can change how that block gets rendered by modifying the content object passed in.
Available blocks in `pad.html` are:
* `htmlHead` - after `<html>` and immediately before the title tag
* `styles` - the style `<link>`s
* `body` - the contents of the body tag
* `editbarMenuLeft` - the left tool bar (consider using the toolbar controller instead of manually adding html here)
* `editbarMenuRight` - right tool bar
* `afterEditbar` - allows you to add stuff immediately after the toolbar
* `userlist` - the contents of the userlist dropdown
* `loading` - the initial loading message
* `mySettings` - the left column of the settings dropdown ("My view"); intended for adding checkboxes only
* `mySettings.dropdowns` - add your dropdown settings here
* `globalSettings` - the right column of the settings dropdown ("Global view")
* `importColumn` - import form
* `exportColumn` - export form
* `modals` - Contains all connectivity messages
* `embedPopup` - the embed dropdown
* `scripts` - Add your script tags here, if you really have to (consider use client-side hooks instead)
`timeslider.html` blocks:
* `timesliderStyles`
* `timesliderScripts`
* `timesliderBody`
* `timesliderTop`
* `timesliderEditbarRight`
* `modals`
`index.html` blocks:
* `indexCustomStyles` - contains the `index.css``<link>` tag, allows you to add your own or to customize the one provided by the active skin
* `indexWrapper` - contains the form for creating new pads
* `indexCustomScripts` - contains the `index.js``<script>` tag, allows you to add your own or to customize the one provided by the active skin
* `indexCustomInlineScripts` - contains the inline `<script>` of home page, allows you to customize `go2Name()`, `go2Random()` or `randomPadName()` functions
## padInitToolbar
Called from: src/node/hooks/express/specialpages.js
Things in context:
1. toolbar - the toolbar controller that will render the toolbar eventually
Here you can add custom toolbar items that will be available in the toolbar config in `settings.json`. For more about the toolbar controller see the API section.
Usage examples:
* https://github.com/tiblu/ep_authorship_toggle
## onAccessCheck
Called from: src/node/db/SecurityManager.js
Things in context:
1. padID - the pad the user wants to access
2. password - the password the user has given to access the pad
3. token - the token of the author
4. sessionCookie - the session the use has
This hook gets called when the access to the concrete pad is being checked. Return `false` to deny access.
## padCreate
Called from: src/node/db/Pad.js
Things in context:
1. pad - the pad instance
2. author - the id of the author who created the pad
This hook gets called when a new pad was created.
## padLoad
Called from: src/node/db/Pad.js
Things in context:
1. pad - the pad instance
This hook gets called when a pad was loaded. If a new pad was created and loaded this event will be emitted too.
## padUpdate
Called from: src/node/db/Pad.js
Things in context:
1. pad - the pad instance
2. author - the id of the author who updated the pad
This hook gets called when an existing pad was updated.
## padCopy
Called from: src/node/db/Pad.js
Things in context:
1. originalPad - the source pad instance
2. destinationID - the id of the pad copied from originalPad
This hook gets called when an existing pad was copied.
Usage examples:
* https://github.com/ether/ep_comments
## padRemove
Called from: src/node/db/Pad.js
Things in context:
1. padID
This hook gets called when an existing pad was removed/deleted.
Usage examples:
* https://github.com/ether/ep_comments
## socketio
Called from: src/node/hooks/express/socketio.js
Things in context:
1. app - the application object
2. io - the socketio object
3. server - the http server object
I have no idea what this is useful for, someone else will have to add this description.
## authorize
Called from: src/node/hooks/express/webaccess.js
Things in context:
1. req - the request object
2. res - the response object
3. next - ?
4. resource - the path being accessed
This hook is called to handle authorization. It is especially useful for
controlling access to specific paths.
A plugin's authorize function is typically called twice for each access: once
before authentication and again after. Specifically, it is called if all of the
following are true:
* The request is not for static content or an API endpoint. (Requests for static
content and API endpoints are always authorized, even if unauthenticated.)
* Either authentication has not yet been performed (`context.req.session.user`
is undefined) or the user has successfully authenticated
(`context.req.session.user` is an object containing user-specific settings).
* If the user has successfully authenticated, the user is not an admin. (Admin
users are always authorized.)
* Either the request is for an `/admin` page or the `requireAuthentication`
setting is true.
* Either the request is for an `/admin` page, or the user has not yet
authenticated, or the user has authenticated and the `requireAuthorization`
setting is true.
* For pre-authentication invocations of a plugin's authorize function
(`context.req.session.user` is undefined), an authorize function from a
different plugin has not already caused the pre-authentication authorization
to pass or fail.
* For post-authentication invocations of a plugin's authorize function
(`context.req.session.user` is an object), an authorize function from a
different plugin has not already caused the post-authentication authorization
to pass or fail.
For pre-authentication invocations of your authorize function, you can pass the
following values to the provided callback:
* `[true]` or `['create']` will immediately grant access without requiring the
user to authenticate.
* `[false]` will trigger authentication unless authentication is not required.
* `[]` or `undefined` will defer the decision to the next authorization plugin
(if any, otherwise it is the same as calling with `[false]`).
**WARNING:** Your authorize function can be called for an `/admin` page even if
the user has not yet authenticated. It is your responsibility to fail or defer
authorization if you do not want to grant admin privileges to the general
public.
For post-authentication invocations of your authorize function, you can pass the
following values to the provided callback:
* `[true]` or `['create']` will grant access.
* `[false]` will deny access.
* `[]` or `undefined` will defer the authorization decision to the next
authorization plugin (if any, otherwise deny).
Example:
```
exports.authorize = (hookName, context, cb) => {
const user = context.req.session.user;
if (!user) {
// The user has not yet authenticated so defer the pre-authentication
// authorization decision to the next plugin.
return cb([]);
}
const path = context.req.path; // or context.resource
if (isExplicitlyProhibited(user, path)) return cb([false]);
if (isExplicitlyAllowed(user, path)) return cb([true]);
return cb([]); // Let the next authorization plugin decide
};
```
## authenticate
Called from: src/node/hooks/express/webaccess.js
Things in context:
1. req - the request object
2. res - the response object
3. users - the users object from settings.json (possibly modified by plugins)
4. next - ?
5. username - the username used (optional)
6. password - the password used (optional)
This hook is called to handle authentication.
Plugins that supply an authenticate function should probably also supply an
authFailure function unless falling back to HTTP basic authentication is
appropriate upon authentication failure.
This hook is only called if either the `requireAuthentication` setting is true
or the request is for an `/admin` page.
Calling the provided callback with `[true]` or `[false]` will cause
authentication to succeed or fail, respectively. Calling the callback with `[]`
or `undefined` will defer the authentication decision to the next authentication
plugin (if any, otherwise fall back to HTTP basic authentication).
If you wish to provide a mix of restricted and anonymous access (e.g., some pads
are private, others are public), you can "authenticate" (as a guest account)
users that have not yet logged in, and rely on other hooks (e.g., authorize,
onAccessCheck, handleMessageSecurity) to authorize specific privileged actions.
If authentication is successful, the authenticate function MUST set
`context.req.session.user` to the user's settings object. The `username`
property of this object should be set to the user's username. The settings
object should come from global settings (`context.users[username]`).
This hook will allow a plug-in developer to modify the file name of an exported pad. This is useful if you want to export a pad under another name and/or hide the padId under export. Note that the doctype or file extension cannot be modified for security reasons.
This hook will allow a plug-in developer to include more properties and attributes to support during HTML Export. If tags are stored as `['color', 'red']` on the attribute pool, use `exportHtmlAdditionalTagsWithData` instead. An Array should be returned.
Identical to `exportHtmlAdditionalTags`, but for tags that are stored with a specific value (not simply `true`) on the attribute pool. For example `['color', 'red']`, instead of `['bold', true]`. This hook will allow a plug-in developer to include more properties and attributes to support during HTML Export. An Array of arrays should be returned. The exported HTML will contain tags like `<span data-color="red">` for the content where attributes are `['color', 'red']`.
The API gives another web application control of the pads. The basic functions are
* create/delete pads
@ -9,15 +9,15 @@ The API gives another web application control of the pads. The basic functions a
The API is designed in a way, so you can reuse your existing user system with their permissions, and map it to Etherpad. Means: Your web application still has to do authentication, but you can tell Etherpad via the api, which visitors should get which permissions. This allows Etherpad to fit into any web application and extend it with real-time functionality. You can embed the pads via an iframe into your website.
Take a look at [HTTP API client libraries](https://github.com/ether/etherpad-lite/wiki/HTTP-API-client-libraries) to check if a library in your favorite programming language is available.
Take a look at https://github.com/ether/etherpad-lite/wiki/HTTP-API-client-libraries[HTTP API client libraries] to check if a library in your favorite programming language is available.
### OpenAPI
==== OpenAPI
OpenAPI (formerly swagger) definitions are exposed under `/api/openapi.json` (latest) and `/api/{version}/openapi.json`. You can use official tools like [Swagger Editor](https://editor.swagger.io/) to view and explore them.
OpenAPI (formerly swagger) definitions are exposed under `/api/openapi.json` (latest) and `/api/{version}/openapi.json`. You can use official tools like https://editor.swagger.io/[Swagger Editor] to view and explore them.
## Examples
=== Examples
### Example 1
==== Example 1
A portal (such as WordPress) wants to give a user access to a new pad. Let's assume the user have the internal id 7 and his name is michael.
@ -47,7 +47,7 @@ Portal starts the session for the user on the group:
Portal places the cookie "sessionID" with the given value on the client and creates an iframe including the pad.
### Example 2
==== Example 2
A portal (such as WordPress) wants to transform the contents of a pad that multiple admins edited into a blog post.
@ -62,14 +62,14 @@ Portal submits content into new blog post
> Portal.AddNewBlog(content)
>
## Usage
=== Usage
### API version
The latest version is `1.2.14`
==== API version
The latest version is `1.2.15`
The current version can be queried via /api.
### Request Format
==== Request Format
The API is accessible via HTTP. Starting from **1.8**, API endpoints can be invoked indifferently via GET or POST.
@ -80,30 +80,37 @@ When invoking via GET (mandatory until **1.7.5** included), parameters must be i
Starting from Etherpad **1.8** it is also possible to invoke the HTTP API via POST. In this case, querystring parameters will still be accepted, but **any parameter with the same name sent via POST will take precedence**. If you need to send large chunks of text (for example, for `setText()`) it is advisable to invoke via POST.
Example with cURL using GET (toy example, no encoding):
Example with cURL using GET (better example, encodes text):
```
[source,bash]
----
curl "http://pad.domain/api/1/setText?apikey=secret&padID=padname" --get --data-urlencode "text=Text sent via GET with proper encoding. For big documents, please use POST"
```
----
Example with cURL using POST:
```
curl "http://pad.domain/api/1/setText?apikey=secret&padID=padname" --data-urlencode "text=Text sent via POST with proper encoding. For big texts (>8 KB), use this method"
```
### Response Format
[source,bash]
----
curl "http://pad.domain/api/1/setText?apikey=secret&padID=padname" --data-urlencode "text=Text sent via POST with proper encoding. For big texts (>8 KB), use this method"
----
==== Response Format
Responses are valid JSON in the following format:
```json
[source,jsonlines]
----
{
"code": number,
"message": string,
"data": obj
}
```
----
* **code** a return code
* **0** everything ok
@ -114,11 +121,11 @@ Responses are valid JSON in the following format:
* **message** a status message. It's ok if everything is fine, else it contains an error message
* **groupID** a string, the unique id of a group. Format is g.16RANDOMCHARS, for example g.s8oes9dhwrvt0zif
* **sessionID** a string, the unique id of a session. Format is s.16RANDOMCHARS, for example s.s8oes9dhwrvt0zif
@ -126,230 +133,248 @@ Responses are valid JSON in the following format:
* **readOnlyID** a string, the unique id of a readonly relation to a pad. Format is r.16RANDOMCHARS, for example r.s8oes9dhwrvt0zif
* **padID** a string, format is GROUPID$PADNAME, for example the pad test of group g.s8oes9dhwrvt0zif has padID g.s8oes9dhwrvt0zif$test
### Authentication
==== Authentication
Authentication works via a token that is sent with each request as a post parameter. There is a single token per Etherpad deployment. This token will be random string, generated by Etherpad at the first start. It will be saved in APIKEY.txt in the root folder of Etherpad. Only Etherpad and the requesting application knows this key. Token management will not be exposed through this API.
### Node Interoperability
==== Node Interoperability
All functions will also be available through a node module accessible from other node.js applications.
### JSONP
=== API Methods
The API provides _JSONP_ support to allow requests from a server in a different domain.
Simply add `&jsonp=?` to the API call.
Example usage: https://api.jquery.com/jQuery.getJSON/
## API Methods
### Groups
==== Groups
Pads can belong to a group. The padID of grouppads is starting with a groupID like g.asdfasdfasdfasdf$test
-> can't be deleted cause this would involve scanning all the pads where this author was
### Session
==== Session
Sessions can be created between a group and an author. This allows an author to access more than one group. The sessionID will be set as a cookie to the client and is valid until a certain date. The session cookie can also contain multiple comma-separated sessionIDs, allowing a user to edit pads in different groups at the same time. Only users with a valid session for this group, can access group pads. You can create a session after you authenticated the user at your web application, to give them access to the pads. You should save the sessionID of this session and delete it after the user logged out.
* `{code: 1, message:"padID does not exist", data: null}`
#### setHTML(padID, html)
===== setHTML(padID, html, [authorId])
* API >= 1
* `authorId` in API >= 1.3.0
sets the text of a pad based on HTML, HTML must be well-formed. Malformed HTML will send a warning to the API log.
If `html` is long (>8 KB), please invoke via POST and include `html` parameter in the body of the request, not in the URL (since Etherpad **1.8**).
*Example returns:*
_Example returns:_
* `{code: 0, message:"ok", data: null}`
* `{code: 1, message:"padID does not exist", data: null}`
#### getAttributePool(padID)
===== getAttributePool(padID)
* API >= 1.2.8
returns the attribute pool of a pad
*Example returns:*
_Example returns:_
* `{ "code":0,
"message":"ok",
"data": {
@ -372,12 +397,13 @@ returns the attribute pool of a pad
}`
* `{"code":1,"message":"padID does not exist","data":null}`
#### getRevisionChangeset(padID, [rev])
===== getRevisionChangeset(padID, [rev])
* API >= 1.2.8
get the changeset at a given revision, or last revision if 'rev' is not defined.
*Example returns:*
_Example returns:_
* `{ "code" : 0,
"message" : "ok",
"data" : "Z:1>6b|5+6b$Welcome to Etherpad!\n\nThis pad text is synchronized as you type, so that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents!\n\nGet involved with Etherpad at https://etherpad.org\n"
@ -385,26 +411,30 @@ get the changeset at a given revision, or last revision if 'rev' is not defined.
* `{"code":1,"message":"padID does not exist","data":null}`
* `{"code":1,"message":"rev is higher than the head revision of the pad","data":null}`
#### createDiffHTML(padID, startRev, endRev)
===== createDiffHTML(padID, startRev, endRev)
* API >= 1.2.7
returns an object of diffs from 2 points in a pad
*Example returns:*
_Example returns:_
* `{"code":0,"message":"ok","data":{"html":"<style>\n.authora_HKIv23mEbachFYfH {background-color: #a979d9}\n.authora_n4gEeMLsv1GivNeh {background-color: #a9b5d9}\n.removed {text-decoration: line-through; -ms-filter:'progid:DXImageTransform.Microsoft.Alpha(Opacity=80)'; filter: alpha(opacity=80); opacity: 0.8; }\n</style>Welcome to Etherpad!<br><br>This pad text is synchronized as you type, so that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents!<br><br>Get involved with Etherpad at <a href=\"http://etherpad.org\">http://etherpad.org</a><br><span class=\"authora_HKIv23mEbachFYfH\">aw</span><br><br>","authors":["a.HKIv23mEbachFYfH",""]}}`
* `{"code":4,"message":"no or wrong API Key","data":null}`
#### restoreRevision(padId, rev)
===== restoreRevision(padId, rev, [authorId])
* API >= 1.2.11
* `authorId` in API >= 1.3.0
Restores revision from past as new changeset
*Example returns:*
_Example returns:_
* {code:0, message:"ok", data:null}
* {code: 1, message:"padID does not exist", data: null}
### Chat
#### getChatHistory(padID, [start, end])
==== Chat
===== getChatHistory(padID, [start, end])
* API >= 1.2.7
returns
@ -413,245 +443,251 @@ returns
* the whole chat history, when no extra parameters are given
creates a chat message, saves it to the database and sends it to all connected clients of this pad
*Example returns:*
_Example returns:_
* `{code: 0, message:"ok", data: null}`
* `{code: 1, message:"text is no string", data: null}`
### Pad
=== Pad
Group pads are normal pads, but with the name schema GROUPID$PADNAME. A security manager controls access of them and it's forbidden for normal pads to include a $ in the name.
#### createPad(padID [, text])
==== createPad(padID, [text], [authorId])
* API >= 1
* `authorId` in API >= 1.3.0
creates a new (non-group) pad. Note that if you need to create a group Pad, you should call **createGroupPad**.
You get an error message if you use one of the following characters in the padID: "/", "?", "&" or "#".
*Example returns:*
_Example returns:_
* `{code: 0, message:"ok", data: null}`
* `{code: 1, message:"padID does already exist", data: null}`
* `{code: 1, message:"malformed padID: Remove special characters", data: null}`
`require("ep_etherpad-lite/static/js/plugingfw/plugins").update()` will use npm to list all installed modules and read their ep.json files, registering the contained hooks.
A hook registration is a pair of a hook name and a function reference (filename for require() plus function name)
## hooks.callAll
`require("ep_etherpad-lite/static/js/plugingfw/hooks").callAll("hook_name", {argname:value})` will call all hook functions registered for `hook_name` with `{argname:value}`.
| Session ID of the https://expressjs.com[Express web framework]. When Etherpad is behind a reverse proxy, and an administrator wants to use session stickiness, he may use this cookie. If you are behind a reverse proxy, please remember to set `trustProxy: true` in `settings.json`. Set in https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/node/hooks/express/webaccess.js#L131[webaccess.js#L131].
|language
| en
| example.org
| /
| Session
| false
| true
| The language of the UI (e.g.: `en-GB`, `it`). Set in https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad_editor.js#L111[pad_editor.js#L111].
|prefs / prefsHttp
| %7B%22epThemesExtTheme%22...
| example.org
| /p
| year 3000
| false
| true
| Client-side preferences (e.g.: font family, chat always visible, show authorship colors, ...). Set in https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad_cookie.js#L49[pad_cookie.js#L49]. `prefs` is used if Etherpad is accessed over HTTPS, `prefsHttp` if accessed over HTTP. For more info see https://github.com/ether/etherpad-lite/issues/3179.
|token
| t.tFzkihhhBf4xKEpCK3PU
| example.org
| /
| 60 days
| false
| true
| A random token representing the author, of the form `t.randomstring_of_lenght_20`. The random string is generated by the client, at https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad.js#L55-L66[pad.js#L55-L66]. This cookie is always set by the client at https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad.js#L153-L158[pad.js#L153-L158] without any solicitation from the server. It is used for all the pads accessed via the web UI (not used for the HTTP API). On the server side, its value is accessed at https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/node/db/SecurityManager.js#L33[SecurityManager.js#L33].
|===
For more info, visit the related discussion at https://github.com/ether/etherpad-lite/issues/3563.
Etherpad HTTP API clients may make use (if they choose so) to send another cookie:
[cols="1,1,1"]
|===
| Name
| Sample value
| Domain
| Usage description
| sessionID
| s.1c70968b333b25476a2c7bdd0e0bed17
| example.org
| Sessions can be created between a group and an author. This allows an author to access more than one group. The sessionID will be set as a cookie to the client and is valid until a certain date. The session cookie can also contain multiple comma-separated sessionIDs, allowing a user to edit pads in different groups at the same time. More info - https://github.com/ether/etherpad-lite/blob/develop/doc/api/http_api.md#session
|express_sid | s%3A7yCNjRmTW8ylGQ53I2IhOwYF9... | example.org | / | Session | true | true | Session ID of the [Express web framework](https://expressjs.com). When Etherpad is behind a reverse proxy, and an administrator wants to use session stickiness, he may use this cookie. If you are behind a reverse proxy, please remember to set `trustProxy: true` in `settings.json`. Set in [webaccess.js#L131](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/node/hooks/express/webaccess.js#L131). |
|language | en | example.org | / | Session | false | true | The language of the UI (e.g.: `en-GB`, `it`). Set in [pad_editor.js#L111](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad_editor.js#L111). |
|prefs / prefsHttp| %7B%22epThemesExtTheme%22... | example.org | /p | year 3000 | false | true | Client-side preferences (e.g.: font family, chat always visible, show authorship colors, ...). Set in [pad_cookie.js#L49](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad_cookie.js#L49). `prefs` is used if Etherpad is accessed over HTTPS, `prefsHttp` if accessed over HTTP. For more info see https://github.com/ether/etherpad-lite/issues/3179. |
|token | t.tFzkihhhBf4xKEpCK3PU | example.org | / | 60 days | false | true | A random token representing the author, of the form `t.randomstring_of_lenght_20`. The random string is generated by the client, at ([pad.js#L55-L66](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad.js#L55-L66)). This cookie is always set by the client (at [pad.js#L153-L158](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/static/js/pad.js#L153-L158)) without any solicitation from the server. It is used for all the pads accessed via the web UI (not used for the HTTP API). On the server side, its value is accessed at [SecurityManager.js#L33](https://github.com/ether/etherpad-lite/blob/01497aa399690e44393e91c19917d11d025df71b/src/node/db/SecurityManager.js#L33).|
For more info, visit the related discussion at https://github.com/ether/etherpad-lite/issues/3563.
Etherpad HTTP API clients may make use (if they choose so) to send another cookie:
| Name | Sample value | Domain | Usage description |
|sessionID | s.1c70968b333b25476a2c7bdd0e0bed17 | example.org | Sessions can be created between a group and an author. This allows an author to access more than one group. The sessionID will be set as a cookie to the client and is valid until a certain date. The session cookie can also contain multiple comma-separated sessionIDs, allowing a user to edit pads in different groups at the same time. More info - https://github.com/ether/etherpad-lite/blob/develop/doc/api/http_api.md#session |
The official Docker image is available on https://hub.docker.com/r/etherpad/etherpad.
=== Downloading from Docker Hub
If you are ok downloading a https://hub.docker.com/r/etherpad/etherpad[prebuilt image from Docker Hub], these are the commands:
[source, bash]
----
# gets the latest published version
docker pull etherpad/etherpad
# gets a specific version
docker pull etherpad/etherpad:1.8.0
----
=== Build a personalized container
If you want to use a personalized settings file, **you will have to rebuild your image**.
All of the following instructions are as a member of the `docker` group.
By default, the Etherpad Docker image is built and run in `production` mode: no development dependencies are installed, and asset bundling speeds up page load time.
==== Rebuilding with custom settings
Edit `<BASEDIR>/settings.json.docker` at your will. When rebuilding the image, this file will be copied inside your image and renamed to `settings.json`.
**Each configuration parameter can also be set via an environment variable**, using the syntax `"${ENV_VAR}"` or `"${ENV_VAR:default_value}"`. For details, refer to `settings.json.template`.
==== Rebuilding including some plugins
If you want to install some plugins in your container, it is sufficient to list them in the ETHERPAD_PLUGINS build variable.
The variable value has to be a space separated, double quoted list of plugin names (see examples).
Some plugins will need personalized settings. Just refer to the previous section, and include them in your custom `settings.json.docker`.
==== Rebuilding including export functionality for DOC/PDF/ODT
If you want to be able to export your pads to DOC/PDF/ODT files, you can install
either Abiword or Libreoffice via setting a build variable.
===== Via Abiword
For installing Abiword, set the `INSTALL_ABIWORD` build variable to any value.
Also, you will need to configure the path to the abiword executable
via setting the `abiword` property in `<BASEDIR>/settings.json.docker` to
`/usr/bin/abiword` or via setting the environment variable `ABIWORD` to
`/usr/bin/abiword`.
===== Via Libreoffice
For installing Libreoffice instead, set the `INSTALL_SOFFICE` build variable
to any value.
Also, you will need to configure the path to the libreoffice executable
via setting the `soffice` property in `<BASEDIR>/settings.json.docker` to
`/usr/bin/soffice` or via setting the environment variable `SOFFICE` to
`/usr/bin/soffice`.
==== Examples
Build a Docker image from the currently checked-out code:
docker run --detach --publish <DESIRED_PORT>:9001 <YOUR_USERNAME>/etherpad
----
And point your browser to `http://<YOUR_IP>:<DESIRED_PORT>`
=== Options available by default
The `settings.json.docker` available by default allows to control almost every setting via environment variables.
==== General
[cols="1,1,1"]
|===
| Variable
| Description
| Default
| `TITLE`
| The name of the instance
| `Etherpad`
| `FAVICON`
| favicon default name, or a fully specified URL to your own favicon
| `favicon.ico`
| `DEFAULT_PAD_TEXT`
| The default text of a pad
| `Welcome to Etherpad! This pad text is synchronized as you type, so that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents! Get involved with Etherpad at https://etherpad.org`
| `IP`
| IP which etherpad should bind at. Change to `::` for IPv6
| `0.0.0.0`
| `PORT`
| port which etherpad should bind at
| `9001`
| `ADMIN_PASSWORD`
| the password for the `admin` user (leave unspecified if you do not want to create it)
|
| `USER_PASSWORD`
| the password for the first user `user` (leave unspecified if you do not want to create it)
|
|===
==== Database
[cols="1,1,1"]
|===
| Variable
| Description
| Default
| `DB_TYPE` | a database supported by https://www.npmjs.com/package/ueberdb2 | not set, thus will fall back to `DirtyDB` (please choose one instead)
| `DB_HOST` | the host of the database
|
| `DB_PORT`
| the port of the database
|
| `DB_NAME`
| the database name
|
| `DB_USER`
| a database user with sufficient permissions to create tables
|
| `DB_PASS`
| the password for the database username
|
| `DB_CHARSET`
| the character set for the tables (only required for MySQL)
|
| `DB_FILENAME`
| in case `DB_TYPE` is `DirtyDB` or `sqlite`, the database file.
| `var/dirty.db`, `var/etherpad.sq3`
|===
If your database needs additional settings, you will have to use a personalized `settings.json.docker` and rebuild the container (or otherwise put the updated `settings.json` inside your image).
==== Pad Options
[cols="1,1,1"]
|===
| Variable
| Description
| Default
| `PAD_OPTIONS_NO_COLORS`
|
| `false`
| `PAD_OPTIONS_SHOW_CONTROLS`
|
| `true`
| `PAD_OPTIONS_SHOW_CHAT`
|
| `true`
| `PAD_OPTIONS_SHOW_LINE_NUMBERS`
|
| `true`
| `PAD_OPTIONS_USE_MONOSPACE_FONT`
|
| `false`
| `PAD_OPTIONS_USER_NAME`
|
| `null`
| `PAD_OPTIONS_USER_COLOR`
|
| `null`
| `PAD_OPTIONS_RTL`
|
| `false`
| `PAD_OPTIONS_ALWAYS_SHOW_CHAT`
|
| `false`
| `PAD_OPTIONS_CHAT_AND_USERS`
|
| `false`
| `PAD_OPTIONS_LANG`
|
| `null`
|===
==== Shortcuts
[cols="1,1,1"]
|===
| Variable
| Description
| Default
| `PAD_SHORTCUTS_ENABLED_ALT_F9`
| focus on the File Menu and/or editbar
| `true`
| `PAD_SHORTCUTS_ENABLED_ALT_C`
| focus on the Chat window
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_S`
| save a revision
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_Z`
| undo/redo
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_Y`
| redo
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_I`
| italic
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_B`
| bold
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_U`
| underline
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_H`
| backspace
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_5`
| strike through
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_SHIFT_1`
| ordered list
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_SHIFT_2`
| shows a gritter popup showing a line author
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_SHIFT_L`
| unordered list
| `true`
| `PAD_SHORTCUTS_ENABLED_CMD_SHIFT_N`
| ordered list
|`true`
| `PAD_SHORTCUTS_ENABLED_CMD_SHIFT_C`
| clear authorship
| `true`
| `PAD_SHORTCUTS_ENABLED_DELETE`
|
| `true`
| `PAD_SHORTCUTS_ENABLED_RETURN`
|
| `true`
| `PAD_SHORTCUTS_ENABLED_ESC`
| in mozilla versions 14-19 avoid reconnecting pad
| `true`
| `PAD_SHORTCUTS_ENABLED_TAB`
| indent
| `true`
| `PAD_SHORTCUTS_ENABLED_CTRL_HOME`
| scroll to top of pad
| `true`
| `PAD_SHORTCUTS_ENABLED_PAGE_UP`
|
| `true`
| `PAD_SHORTCUTS_ENABLED_PAGE_DOWN`
|
| `true`
|===
==== Skins
You can use the UI skin variants builder at `/p/test#skinvariantsbuilder`
For the colibris skin only, you can choose how to render the three main containers:
* toolbar (top menu with icons)
* editor (containing the text of the pad)
* background (area outside of editor, mostly visible when using page style)
For each of the 3 containers you can choose 4 color combinations:
* super-light
* light
* dark
* super-dark
For the editor container, you can also make it full width by adding `full-width-editor` variant (by default editor is rendered as a page, with a max-width of 900px).
[cols="1,1,1"]
|===
| Variable
| Description
| Default
| `SKIN_NAME`
| either `no-skin`, `colibris` or an existing directory under `src/static/skins`
| valid values are `DEBUG`, `INFO`, `WARN` and `ERROR` | `INFO`
| `DISABLE_IP_LOGGING`
| Privacy: disable IP logging
| `false`
|===
==== Advanced
[cols="1,1,1"]
|===
| Variable
| Description
| Default
|`COOKIE_KEY_ROTATION_INTERVAL`
|How often (ms) to rotate in a new secret for signing cookies
|`86400000` (1 day)
| `COOKIE_SAME_SITE`
| Value of the SameSite cookie property.
| `"Lax"`
| `COOKIE_SESSION_LIFETIME`
| How long (ms) a user can be away before they must log in again.
| `864000000` (10 days)
| `COOKIE_SESSION_REFRESH_INTERVAL`
| How often (ms) to write the latest cookie expiration time.
| `86400000` (1 day)
| `SHOW_SETTINGS_IN_ADMIN_PAGE`
| hide/show the settings.json in admin page
| `true`
| `TRUST_PROXY`
| set to `true` if you are using a reverse proxy in front of Etherpad (for example: Traefik for SSL termination via Let's Encrypt). This will affect security and correctness of the logs if not done
| `false`
| `IMPORT_MAX_FILE_SIZE`
| maximum allowed file size when importing a pad, in bytes.
| `52428800` (50 MB)
| `IMPORT_EXPORT_MAX_REQ_PER_IP`
| maximum number of import/export calls per IP.
| `10`
| `IMPORT_EXPORT_RATE_LIMIT_WINDOW`
| the call rate for import/export requests will be estimated in this time window (in milliseconds)
| `90000`
| `COMMIT_RATE_LIMIT_DURATION`
| duration of the rate limit window for commits by individual users/IPs (in seconds) | `1`
| `COMMIT_RATE_LIMIT_POINTS`
| maximum number of changes per IP to allow during the rate limit window
| `10`
| `SUPPRESS_ERRORS_IN_PAD_TEXT`
| Should we suppress errors from being visible in the default Pad Text?
| `false
| `REQUIRE_SESSION`
| If this option is enabled, a user must have a session to access pads. This effectively allows only group pads to be accessed.
| `false`
| `EDIT_ONLY`
| Users may edit pads but not create new ones. Pad creation is only via the API. This applies both to group pads and regular pads.
| `false`
| `MINIFY`
| If true, all css & js will be minified before sending to the client. This will improve the loading performance massively, but makes it difficult to debug the javascript/css
| `true`
| `MAX_AGE`
| How long may clients use served javascript code (in seconds)? Not setting this may cause problems during deployment. Set to 0 to disable caching.
| `21600` (6 hours)
| `ABIWORD`
| Absolute path to the Abiword executable. Abiword is needed to get advanced import/export features of pads. Setting it to null disables Abiword and will only allow plain text and HTML import/exports.
| `null`
| `SOFFICE`
| This is the absolute path to the soffice executable. LibreOffice can be used in lieu of Abiword to export pads. Setting it to null disables LibreOffice exporting.
| `null`
| `TIDY_HTML`
| Path to the Tidy executable. Tidy is used to improve the quality of exported pads. Setting it to null disables Tidy.
| `null`
| `ALLOW_UNKNOWN_FILE_ENDS`
| Allow import of file types other than the supported ones: txt, doc, docx, rtf, odt, html & htm
| `true`
| `REQUIRE_AUTHENTICATION`
| This setting is used if you require authentication of all users. Note: "/admin" always requires authentication.
| `false`
| `REQUIRE_AUTHORIZATION`
| Require authorization by a module, or a user with is_admin set, see below.
| `false`
| `AUTOMATIC_RECONNECTION_TIMEOUT`
| Time (in seconds) to automatically reconnect pad when a "Force reconnect" message is shown to user. Set to 0 to disable automatic reconnection.
| `0`
| `FOCUS_LINE_PERCENTAGE_ABOVE`
| Percentage of viewport height to be additionally scrolled. e.g. 0.5, to place caret line in the middle of viewport, when user edits a line above of the viewport. Set to 0 to disable extra scrolling
| `0`
| `FOCUS_LINE_PERCENTAGE_BELOW`
| Percentage of viewport height to be additionally scrolled. e.g. 0.5, to place caret line in the middle of viewport, when user edits a line below of the viewport. Set to 0 to disable extra scrolling
| `0`
| `FOCUS_LINE_PERCENTAGE_ARROW_UP`
| Percentage of viewport height to be additionally scrolled when user presses arrow up in the line of the top of the viewport. Set to 0 to let the scroll to be handled as default by Etherpad
| `0`
| `FOCUS_LINE_DURATION`
| Time (in milliseconds) used to animate the scroll transition. Set to 0 to disable animation
| `0`
| `FOCUS_LINE_CARET_SCROLL`
| Flag to control if it should scroll when user places the caret in the last line of the viewport
| `false`
| `SOCKETIO_MAX_HTTP_BUFFER_SIZE`
| The maximum size (in bytes) of a single message accepted via Socket.IO. If a client sends a larger message, its connection gets closed to prevent DoS (memory exhaustion) attacks.
| `10000`
| `LOAD_TEST`
| Allow Load Testing tools to hit the Etherpad Instance. WARNING: this will disable security on the instance.
| `false`
| `DUMP_ON_UNCLEAN_EXIT`
| Enable dumping objects preventing a clean exit of Node.js. WARNING: this has a significant performance impact.
| `false`
| `EXPOSE_VERSION`
| Expose Etherpad version in the web interface and in the Server http header. Do not enable on production machines.
| `false`
|===
==== Examples
Use a Postgres database, no admin user enabled:
[source,bash]
----
docker run -d \
--name etherpad \
-p 9001:9001 \
-e 'DB_TYPE=postgres' \
-e 'DB_HOST=db.local' \
-e 'DB_PORT=4321' \
-e 'DB_NAME=etherpad' \
-e 'DB_USER=dbusername' \
-e 'DB_PASS=mypassword' \
etherpad/etherpad
----
Run enabling the administrative user `admin`:
[source,bash]
----
docker run -d \
--name etherpad \
-p 9001:9001 \
-e 'ADMIN_PASSWORD=supersecret' \
etherpad/etherpad
----
Run a test instance running DirtyDB on a persistent volume:
The official Docker image is available on https://hub.docker.com/r/etherpad/etherpad.
## Downloading from Docker Hub
If you are ok downloading a [prebuilt image from Docker Hub](https://hub.docker.com/r/etherpad/etherpad), these are the commands:
```bash
# gets the latest published version
docker pull etherpad/etherpad
# gets a specific version
docker pull etherpad/etherpad:1.8.0
```
## Build a personalized container
If you want to use a personalized settings file, **you will have to rebuild your image**.
All of the following instructions are as a member of the `docker` group.
By default, the Etherpad Docker image is built and run in `production` mode: no development dependencies are installed, and asset bundling speeds up page load time.
### Rebuilding with custom settings
Edit `<BASEDIR>/settings.json.docker` at your will. When rebuilding the image, this file will be copied inside your image and renamed to `setting.json`.
**Each configuration parameter can also be set via an environment variable**, using the syntax `"${ENV_VAR}"` or `"${ENV_VAR:default_value}"`. For details, refer to `settings.json.template`.
### Rebuilding including some plugins
If you want to install some plugins in your container, it is sufficient to list them in the ETHERPAD_PLUGINS build variable.
The variable value has to be a space separated, double quoted list of plugin names (see examples).
Some plugins will need personalized settings. Just refer to the previous section, and include them in your custom `settings.json.docker`.
### Examples
Build a Docker image from the currently checked-out code:
| `TITLE` | The name of the instance | `Etherpad` |
| `FAVICON` | favicon default name, or a fully specified URL to your own favicon | `favicon.ico` |
| `DEFAULT_PAD_TEXT` | The default text of a pad | `Welcome to Etherpad! This pad text is synchronized as you type, so that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents! Get involved with Etherpad at https://etherpad.org` |
| `IP` | IP which etherpad should bind at. Change to `::` for IPv6 | `0.0.0.0` |
| `PORT` | port which etherpad should bind at | `9001` |
| `ADMIN_PASSWORD` | the password for the `admin` user (leave unspecified if you do not want to create it) | |
| `USER_PASSWORD` | the password for the first user `user` (leave unspecified if you do not want to create it) | |
| `DB_TYPE` | a database supported by https://www.npmjs.com/package/ueberdb2 | not set, thus will fall back to `DirtyDB` (please choose one instead) |
| `DB_HOST` | the host of the database | |
| `DB_PORT` | the port of the database | |
| `DB_NAME` | the database name | |
| `DB_USER` | a database user with sufficient permissions to create tables | |
| `DB_PASS` | the password for the database username | |
| `DB_CHARSET` | the character set for the tables (only required for MySQL) | |
| `DB_FILENAME` | in case `DB_TYPE` is `DirtyDB`, the database filename. | `var/dirty.db` |
If your database needs additional settings, you will have to use a personalized `settings.json.docker` and rebuild the container (or otherwise put the updated `settings.json` inside your image).
| `PAD_SHORTCUTS_ENABLED_ESC` | in mozilla versions 14-19 avoid reconnecting pad | `true` |
| `PAD_SHORTCUTS_ENABLED_TAB` | indent | `true` |
| `PAD_SHORTCUTS_ENABLED_CTRL_HOME` | scroll to top of pad | `true` |
| `PAD_SHORTCUTS_ENABLED_PAGE_UP` | | `true` |
| `PAD_SHORTCUTS_ENABLED_PAGE_DOWN` | | `true` |
### Skins
You can use the UI skin variants builder at `/p/test#skinvariantsbuilder`
For the colibris skin only, you can choose how to render the three main containers:
* toolbar (top menu with icons)
* editor (containing the text of the pad)
* background (area outside of editor, mostly visible when using page style)
For each of the 3 containers you can choose 4 color combinations:
* super-light
* light
* dark
* super-dark
For the editor container, you can also make it full width by adding `full-width-editor` variant (by default editor is rendered as a page, with a max-width of 900px).
| `SHOW_SETTINGS_IN_ADMIN_PAGE` | hide/show the settings.json in admin page | `true` |
| `TRUST_PROXY` | set to `true` if you are using a reverse proxy in front of Etherpad (for example: Traefik for SSL termination via Let's Encrypt). This will affect security and correctness of the logs if not done | `false` |
| `IMPORT_MAX_FILE_SIZE` | maximum allowed file size when importing a pad, in bytes. | `52428800` (50 MB) |
| `IMPORT_EXPORT_MAX_REQ_PER_IP` | maximum number of import/export calls per IP. | `10` |
| `IMPORT_EXPORT_RATE_LIMIT_WINDOW` | the call rate for import/export requests will be estimated in this time window (in milliseconds) | `90000` |
| `SUPPRESS_ERRORS_IN_PAD_TEXT` | Should we suppress errors from being visible in the default Pad Text? | `false` |
| `REQUIRE_SESSION` | If this option is enabled, a user must have a session to access pads. This effectively allows only group pads to be accessed. | `false` |
| `EDIT_ONLY` | Users may edit pads but not create new ones. Pad creation is only via the API. This applies both to group pads and regular pads. | `false` |
| `SESSION_NO_PASSWORD` | If set to true, those users who have a valid session will automatically be granted access to password protected pads. | `false` |
| `MINIFY` | If true, all css & js will be minified before sending to the client. This will improve the loading performance massively, but makes it difficult to debug the javascript/css | `true` |
| `MAX_AGE` | How long may clients use served javascript code (in seconds)? Not setting this may cause problems during deployment. Set to 0 to disable caching. | `21600` (6 hours) |
| `ABIWORD` | Absolute path to the Abiword executable. Abiword is needed to get advanced import/export features of pads. Setting it to null disables Abiword and will only allow plain text and HTML import/exports. | `null` |
| `SOFFICE` | This is the absolute path to the soffice executable. LibreOffice can be used in lieu of Abiword to export pads. Setting it to null disables LibreOffice exporting. | `null` |
| `TIDY_HTML` | Path to the Tidy executable. Tidy is used to improve the quality of exported pads. Setting it to null disables Tidy. | `null` |
| `ALLOW_UNKNOWN_FILE_ENDS` | Allow import of file types other than the supported ones: txt, doc, docx, rtf, odt, html & htm | `true` |
| `REQUIRE_AUTHENTICATION` | This setting is used if you require authentication of all users. Note: "/admin" always requires authentication. | `false` |
| `REQUIRE_AUTHORIZATION` | Require authorization by a module, or a user with is_admin set, see below. | `false` |
| `AUTOMATIC_RECONNECTION_TIMEOUT` | Time (in seconds) to automatically reconnect pad when a "Force reconnect" message is shown to user. Set to 0 to disable automatic reconnection. | `0` |
| `FOCUS_LINE_PERCENTAGE_ABOVE` | Percentage of viewport height to be additionally scrolled. e.g. 0.5, to place caret line in the middle of viewport, when user edits a line above of the viewport. Set to 0 to disable extra scrolling | `0` |
| `FOCUS_LINE_PERCENTAGE_BELOW` | Percentage of viewport height to be additionally scrolled. e.g. 0.5, to place caret line in the middle of viewport, when user edits a line below of the viewport. Set to 0 to disable extra scrolling | `0` |
| `FOCUS_LINE_PERCENTAGE_ARROW_UP` | Percentage of viewport height to be additionally scrolled when user presses arrow up in the line of the top of the viewport. Set to 0 to let the scroll to be handled as default by Etherpad | `0` |
| `FOCUS_LINE_DURATION` | Time (in milliseconds) used to animate the scroll transition. Set to 0 to disable animation | `0` |
| `FOCUS_LINE_CARET_SCROLL` | Flag to control if it should scroll when user places the caret in the last line of the viewport | `false` |
| `LOAD_TEST` | Allow Load Testing tools to hit the Etherpad Instance. WARNING: this will disable security on the instance. | `false` |
| `EXPOSE_VERSION` | Expose Etherpad version in the web interface and in the Server http header. Do not enable on production machines. | `false` |
### Examples
Use a Postgres database, no admin user enabled:
```shell
docker run -d \
--name etherpad \
-p 9001:9001 \
-e 'DB_TYPE=postgres' \
-e 'DB_HOST=db.local' \
-e 'DB_PORT=4321' \
-e 'DB_NAME=etherpad' \
-e 'DB_USER=dbusername' \
-e 'DB_PASS=mypassword' \
etherpad/etherpad
```
Run enabling the administrative user `admin`:
```shell
docker run -d \
--name etherpad \
-p 9001:9001 \
-e 'ADMIN_PASSWORD=supersecret' \
etherpad/etherpad
```
Run a test instance running DirtyDB on a persistent volume:
Etherpad provides a multi-language user interface, that's apart from your users' content, so users from different countries can collaborate on a single document, while still having the user interface displayed in their mother tongue.
## Translating
=== Translating
We rely on https://translatewiki.net to handle the translation process for us, so if you'd like to help...
4. Choose a target language, you'd like to translate our interface to, and hit `Fetch`
5. Start translating!
Translations will be send back to us regularly and will eventually appear in the next release.
## Implementation
=== Implementation
### Server-side
==== Server-side
`/src/locales` contains files for all supported languages which contain the translated strings. Translation files are simple `*.json` files and look like this:
```json
{ "pad.modals.connected": "Connecté."
, "pad.modals.uderdup": "Ouvrir dans une nouvelle fenêtre."
Each translation consists of a key (the id of the string that is to be translated) and the translated string. Terms in curly braces must not be touched but left as they are, since they represent a dynamically changing part of the string like a variable. Imagine a message welcoming a user: `Welcome, {{userName}}!` would be translated as `Ahoy, {{userName}}!` in pirate.
### Client-side
We use a `language` cookie to save your language settings if you change them. If you don't, we autodetect your locale using information from your browser. Then, the preferred language is fed into a library called [html10n.js](https://github.com/marcelklehr/html10n.js), which loads the appropriate translations and applies them to our templates. Its features include translation params, pluralization, include rules and a nice javascript API.
==== Client-side
We use a `language` cookie to save your language settings if you change them. If you don't, we autodetect your locale using information from your browser. Then, the preferred language is fed into a library called https://github.com/marcelklehr/html10n.js[html10n.js], which loads the appropriate translations and applies them to our templates. Its features include translation params, pluralization, include rules and a nice javascript API.
## Localizing plugins
=== Localizing plugins
### 1. Mark the strings to translate
==== 1. Mark the strings to translate
In the template files of your plugin, change all hardcoded messages/strings...
In the javascript files of your plugin, change all hardcoded messages/strings...
from:
```js
alert ('Chat');
```
to:
```js
alert(window._('pad.chat'));
```
### 2. Create translate files in the locales directory of your plugin
* The name of the file must be the language code of the language it contains translations for (see [supported lang codes](https://joker-x.github.com/languages4translatewiki/test/); e.g. en ? English, es ? Spanish...)
[source,js]
----
alert ('Chat');
----
to:
[source,js]
----
alert(window._('pad.chat'));
----
==== 2. Create translate files in the locales directory of your plugin
* The name of the file must be the language code of the language it contains translations for (see https://joker-x.github.io/languages4translatewiki/test/[supported lang codes]; e.g. en ? English, es ? Spanish...)
* The extension of the file must be `.json`
* The default language is English, so your plugin should always provide `en.json`
* In order to avoid naming conflicts, your message keys should start with the name of your plugin followed by a dot (see below)
*ep_your-plugin/locales/en.json*
```
{ "ep_your-plugin.h1": "Heading 1"
[source, json]
----
{
"ep_your-plugin.h1": "Heading 1"
}
```
----
*ep_your-plugin/locales/es.json*
```
{ "ep_your-plugin.h1": "Título 1"
[source, json]
----
{
"ep_your-plugin.h1": "Título 1"
}
```
----
Every time the http server is started, it will auto-detect your messages and merge them automatically with the core messages.
### Overwrite core messages
==== Overwrite core messages
You can overwrite Etherpad's core messages in your plugin's locale files.
For example, if you want to replace `Chat` with `Notes`, simply add...
*ep_your-plugin/locales/en.json*
```
{ "ep_your-plugin.h1": "Heading 1"
, "pad.chat": "Notes"
[source,json]
----
{
"ep_your-plugin.h1": "Heading 1",
"pad.chat": "Notes"
}
```
----
## Customization for Administrators
=== Customization for Administrators
As an Etherpad administrator, it is possible to overwrite core mesages as well as messages in plugins. These include error messages, labels, and user instructions. Whereas the localization in the source code is in separate files separated by locale, an administrator's custom localizations are in `settings.json` under the `customLocaleStrings` key, with each locale separated by a sub-key underneath.
As an Etherpad administrator, it is possible to overwrite core messages as well as messages in plugins. These include error messages, labels, and user instructions. Whereas the localization in the source code is in separate files separated by locale, an administrator's custom localizations are in `settings.json` under the `customLocaleStrings` key, with each locale separated by a sub-key underneath.
For example, let's say you want to change the text on the "New Pad" button on Etherpad's home page. If you look in `locales/en.json` (or `locales/en-gb.json`) you'll see the key for this text is `"index.newPad"`. You could add the following to `settings.json`:
```
[source,json]
----
"customLocaleStrings": {
"fr": {
"index.newPad": "Créer un document"
@ -111,4 +131,4 @@ For example, let's say you want to change the text on the "New Pad" button on Et
Etherpad allows you to extend its functionality with plugins. A plugin registers hooks (functions) for certain events (thus certain features) in Etherpad-lite to execute its own functionality based on these events.
Publicly available plugins can be found in the npm registry (see <https://npmjs.org>). Etherpad-lite's naming convention for plugins is to prefix your plugins with `ep_`. So, e.g. it's `ep_flubberworms`. Thus you can install plugins from npm, using `npm install ep_flubberworm` in etherpad-lite's root directory.
You can also browse to `http://yourEtherpadInstan.ce/admin/plugins`, which will list all installed plugins and those available on npm. It even provides functionality to search through all available plugins.
## Folder structure
A basic plugin usually has the following folder structure:
```
ep_<plugin>/
| static/
| templates/
| locales/
+ ep.json
+ package.json
```
If your plugin includes client-side hooks, put them in `static/js/`. If you're adding in CSS or image files, you should put those files in `static/css/ `and `static/image/`, respectively, and templates go into `templates/`. Translations go into `locales/`
A Standard directory structure like this makes it easier to navigate through your code. That said, do note, that this is not actually *required* to make your plugin run. If you want to make use of our i18n system, you need to put your translations into `locales/`, though, in order to have them integrated. (See "Localization" for more info on how to localize your plugin)
## Plugin definition
Your plugin definition goes into `ep.json`. In this file you register your hooks, indicate the parts of your plugin and the order of execution. (A documentation of all available events to hook into can be found in chapter [hooks](#all_hooks).)
A hook registration is a pairs of a hook name and a function reference (filename to require() + exported function name)
Etherpad-lite will expect the part of the hook definition before the colon to be a javascript file and will try to require it. The part after the colon is expected to be a valid function identifier of that module. So, you have to export your hooks, using [`module.exports`](https://nodejs.org/docs/latest/api/modules.html#modules_modules) and register it in `ep.json` as `ep_<plugin>/path/to/<file>:FUNCTIONNAME`.
You can omit the `FUNCTIONNAME` part, if the exported function has got the same name as the hook. So `"authorize" : "ep_flubberworm/foo"` will call the function `exports.authorize` in `ep_flubberworm/foo.js`
### Client hooks and server hooks
There are server hooks, which will be executed on the server (e.g. `expressCreateServer`), and there are client hooks, which are executed on the client (e.g. `acePopulateDomLine`). Be sure to not make assumptions about the environment your code is running in, e.g. don't try to access `process`, if you know your code will be run on the client, and likewise, don't try to access `window` on the server...
### Styling
When you install a client-side plugin (e.g. one that implements at least one client-side hook), the plugin name is added to the `class` attribute of the div `#editorcontainerbox` in the main window.
This gives you the opportunity of tuning the appearance of the main UI in your plugin.
For example, this is the markup with no plugins installed:
```html
<divid="editorcontainerbox"class="">
```
and this is the contents after installing `someplugin`:
```html
<divid="editorcontainerbox"class="ep_someplugin">
```
This feature was introduced in Etherpad **1.8**.
### Parts
As your plugins become more and more complex, you will find yourself in the need to manage dependencies between plugins. E.g. you want the hooks of a certain plugin to be executed before (or after) yours. You can also manage these dependencies in your plugin definition file `ep.json`:
Usually a plugin will add only one functionality at a time, so it will probably only use one `part` definition to register its hooks. However, sometimes you have to put different (unrelated) functionalities into one plugin. For this you will want use parts, so other plugins can depend on them.
#### pre/post
The `"pre"` and `"post"` definitions, affect the order in which parts of a plugin are executed. This ensures that plugins and their hooks are executed in the correct order.
`"pre"` lists parts that must be executed *before* the defining part. `"post"` lists parts that must be executed *after* the defining part.
You can, on a basic level, think of this as double-ended dependency listing. If you have a dependency on another plugin, you can make sure it loads before yours by putting it in `"pre"`. If you are setting up things that might need to be used by a plugin later, you can ensure proper order by putting it in `"post"`.
Note that it would be far more sane to use `"pre"` in almost any case, but if you want to change config variables for another plugin, or maybe modify its environment, `"post"` could definitely be useful.
Also, note that dependencies should *also* be listed in your package.json, so they can be `npm install`'d automagically when your plugin gets installed.
## Package definition
Your plugin must also contain a [package definition file](https://docs.npmjs.com/files/package.json), called package.json, in the project root - this file contains various metadata relevant to your plugin, such as the name and version number, author, project hompage, contributors, a short description, etc. If you publish your plugin on npm, these metadata are used for package search etc., but it's necessary for Etherpad-lite plugins, even if you don't publish your plugin.
If your plugin adds or modifies the front end HTML (e.g. adding buttons or changing their functions), you should put the necessary HTML code for such operations in `templates/`, in files of type ".ejs", since Etherpad uses EJS for HTML templating. See the following link for more information about EJS: <https://github.com/visionmedia/ejs>.
## Writing and running front-end tests for your plugin
Etherpad allows you to easily create front-end tests for plugins.
1. Create a new folder
```
%your_plugin%/static/tests/frontend/specs
```
2. Put your spec file in here (Example spec files are visible in %etherpad_root_folder%/frontend/tests/specs)
3. Visit http://yourserver.com/frontend/tests your front-end tests will run.
Etherpad keeps track of the goings-on inside the edit machinery. If you'd like to have a look at this, just point your browser to `/stats`.
We currently measure:
@ -13,6 +14,6 @@ We currently measure:
- http500 (meter)
- memoryUsage (gauge)
Under the hood, we are happy to rely on [measured](https://github.com/felixge/node-measured) for all our metrics needs.
Under the hood, we are happy to rely on https://github.com/felixge/node-measured[measured] for all our metrics needs.
To modify or simply access our stats in your plugin, simply `require('ep_etherpad-lite/stats')` which is a [`measured.Collection`](https://yaorg.github.io/node-measured/packages/measured-core/Collection.html).
To modify or simply access our stats in your plugin, simply `require('ep_etherpad-lite/stats')` which is a https://yaorg.github.io/node-measured/packages/measured-core/Collection.html[`measured.Collection`].
dependabot[bot]
