cal.pub0.org/apps/web/public/robots.txt

User-agent: *
Disallow: /sandbox
Disallow: /api
Disallow: /static/locales

# Specifically allow access to OG Image api, otherwise eg Twitter won't render these images.
Allow: /api/social/og/image
# Specifically allow access to api/app-store to allow search engines to fetch assets
Allow: /api/app-store/
Setup i18n and locale detection (#712) * feat: setup translations * feat: i18n setup * Update pages/settings/profile.tsx Co-authored-by: Alex Johansson <alexander@n1s.se> * fix: abstract locale hook * fix: set default locale if preferred locale is not supported * Revert "fix: set default locale if preferred locale is not supported" This reverts commit e2a3d81371ee02a033520058a1d7d61cffeffc94. * fix: set default locale if preferred locale is not supported * fix: use 1 namespace and remove unnecessary logs * fix: yarn.lock * fix: linting errors Co-authored-by: Peer Richelsen <peeroke@gmail.com> Co-authored-by: Alex Johansson <alexander@n1s.se> 2021-09-23 08:49:17 +00:00			`User-agent: *`
			`Disallow: /sandbox`
			`Disallow: /api`
			`Disallow: /static/locales`
Allow og image api in robots.txt (#5283) Co-authored-by: Peer Richelsen <peeroke@gmail.com> 2022-10-31 09:48:17 +00:00
			`# Specifically allow access to OG Image api, otherwise eg Twitter won't render these images.`
chore: update robots.txt to allow api/app-store (#8023) 2023-04-03 14:43:21 +00:00			`Allow: /api/social/og/image`
			`# Specifically allow access to api/app-store to allow search engines to fetch assets`
			`Allow: /api/app-store/`