SEO, Analytics and Privacy

The sitemap index lists one file per provider and sub-type. On the Lumen Press site, after activating lumen-catalog (which registers lp_book, lp_genre and lp_series as public and queryable), core lists them without any extra code:

Source: Make WordPress Core: New XML Sitemaps Functionality in WordPress 5.5

The output is shown one entry per line; core prints the index on a single line. The XSL stylesheet only makes the file readable in a browser; crawlers ignore it. How the sitemaps behave:

The wp_sitemaps_* filters change what core lists. The Lumen Press catalogue drops the users sub-sitemap (author archives are thin pages on this site), leaves tags out, and adds <lastmod> to every post, page and book:

Source: Make WordPress Core: New XML Sitemaps Functionality in WordPress 5.5

After activating the plugin, the book sub-sitemap carries the modification date:

The other filters, all since 5.5: wp_sitemaps_enabled (return false, for example with __return_false, to switch sitemaps off; the rewrite rules stay, so the URLs answer 404 instead of falling through to a page), wp_sitemaps_post_types (remove post types), wp_sitemaps_posts_query_args, wp_sitemaps_taxonomies_query_args and wp_sitemaps_users_query_args (change the query, for example to exclude IDs), wp_sitemaps_taxonomies_entry, wp_sitemaps_users_entry and wp_sitemaps_index_entry (add fields to one entry), wp_sitemaps_max_urls (URLs per page, per object type), wp_sitemaps_*_pre_max_num_pages (short-circuit the page count) and wp_sitemaps_stylesheet_url with its relatives (change or remove the XSL presentation).

A new kind of content (for example events kept in a custom table) gets its own provider: a class extending WP_Sitemaps_Provider that implements get_url_list() and get_max_num_pages(), registered with wp_register_sitemap_provider( 'lumen-events', $provider ) on init. Content in custom post types needs no provider, as lp_book shows.

When no physical robots.txt exists in the document root, the web server passes the request to index.php, and do_robots() prints the file. On a public site, and then with "Discourage search engines" ticked (the Sitemap: line disappears and the sitemaps answer 404):

Source: Code Reference: do_robots()

Before WordPress 5.3, a private site’s robots.txt contained Disallow: /. That rule stops crawling but not indexing: a blocked URL can still be indexed from links elsewhere, and the crawler never sees the page’s noindex. Since 5.3 core relies on the robots meta tag instead (next section), and the robots.txt stays permissive.

The robots_txt filter changes the output. The Lumen Press plugin asks crawlers to skip its REST routes. Core appends the Sitemap: line from a callback at priority 0, so the plugin’s callback runs at -1 to keep its rule inside the User-agent: * group:

Source: Code Reference: robots_txt hook

Pitfalls:

Since 5.7 the robots meta tag is assembled by the Robots API: wp_robots() runs on wp_head and prints the directives returned by the wp_robots filter. Core’s callbacks: wp_robots_noindex (noindex, nofollow on every front-end page when blog_public is 0), wp_robots_noindex_search (noindex on search results), wp_robots_noindex_embeds (noindex on the /embed/ view of a post), wp_robots_max_image_preview_large (max-image-preview:large on public sites, allowing large image previews) and, on the login screen, wp_robots_sensitive_page (noindex, noarchive; it replaces wp_sensitive_page_meta(), deprecated in 5.7). The helper wp_robots_no_robots returns noindex plus follow or nofollow depending on blog_public; core also adds it for comment reply links (?replytocom=) and unapproved comment previews.

Source: Make WordPress Core: Robots API and max-image-preview directive in WordPress 5.7

The output on the Lumen Press site, first public and then with the Reading setting ticked:

The setting is labelled "Search engine visibility" on Settings > Reading, and the screen itself warns that it is up to search engines to honour the request. While it is ticked, the dashboard’s At a Glance box shows "Search engines discouraged". It is the right switch for staging copies and a site under construction; on staging it is often safer to add HTTP authentication as well, because a page indexed before the switch was ticked can stay in results until it is recrawled. Forgetting to untick it at launch is one of the most common reasons a new WordPress site does not appear in search results; a deployment check of wp option get blog_public catches it.

A plugin adds directives through the same filter. Lumen Press keeps date archives and attachment pages out of the index, while still letting crawlers follow their links:

Source: Code Reference: wp_robots hook

A date archive such as /2026/10/ then prints max-image-preview:large, noindex, follow. Each array key is a directive; a value of true prints the bare name, a string prints name:value. A directive set to false is dropped, which is how a callback removes one that an earlier callback added. Attachment pages themselves can be switched off: since 6.4 new installs set the wp_attachment_pages_enabled option to 0, which redirects attachment URLs to the file, and the noindex above matters only for older sites that keep them.

Block themes always print <title> (core hooks _block_template_render_title_tag() to wp_head when it renders a block template); classic themes declare add_theme_support( 'title-tag' ) and must not hard-code a <title> in header.php. In both cases the text comes from wp_get_document_title(): the post or archive title, the page number for paginated views, and the site name (with the tagline on the front page), joined by a separator.

Three filters change it: pre_get_document_title (return a non-empty string to replace the whole title; SEO plugins use it), document_title_parts (the title, page, tagline and site parts before joining) and document_title_separator (default -, printed as an en dash).

Lumen Press adds the content type to book titles and uses a vertical bar:

Source: Code Reference: wp_get_document_title()

The <head> of a single book, with the default theme Twenty Twenty-Five 1.5 active:

rel_canonical() prints the canonical link on singular views only (posts, pages, lp_book entries), using wp_get_canonical_url(); for paginated posts and comment pages it adds the page number. Archives, the blog index and search results get no canonical link from core, and the get_canonical_url filter changes the URL for singular views. Google treats the canonical link as a strong hint, not a directive, and combines it with redirects, sitemaps and internal links. The shortlink (?p=5) is a convenience for sharing; it redirects to the permalink and has no SEO role.

The search engine shows its own title link, which may differ from the <title>; a title that describes the page and is not stuffed with keywords is the one most likely to be used.

Since 5.3, wp_rel_ugc() runs on pre_comment_content and adds rel="nofollow ugc" to every link a visitor writes in a comment, and the comment author’s website link is printed with rel="external nofollow ugc". ugc tells search engines that the link is user-generated content; nofollow is kept for crawlers that do not know ugc. A comment saved through wp_new_comment() with the body see <a href="https://reviews.example.net/ledger">my review</a> is stored as see <a href="https://reviews.example.net/ledger" rel="nofollow ugc">my review</a>. Source: Make WordPress Core: WP 5.3 introduces new functions to add UGC attribute to links

The attribute is added when the comment is saved through wp_new_comment() (the comment form, the REST API, the XML-RPC API). Comments inserted directly with wp_insert_comment(), including wp comment create, skip the filter and keep their links unchanged.

A new install creates a draft page titled "Privacy Policy" and stores its ID in the wp_page_for_privacy_policy option. Settings > Privacy (options-privacy.php) chooses or creates the page, and its Policy Guide tab (since 5.3) collects suggested text from core and from active plugins that call wp_add_privacy_policy_content(); the guide also flags text from plugins that have been deactivated since. The suggested text is a starting point that the site owner edits, not legal wording. Once the page is published:

Source: WordPress.org Documentation: Settings Privacy Screen

Core then links the page from the login and registration screens, and themes link it with the_privacy_policy_link() or, in block themes, by adding the page to a footer Navigation block. While the selected page is not published, get_privacy_policy_url() returns an empty string and the link functions print nothing.

Core’s own cookies are few: the authentication and wordpress_test_cookie cookies for logged-in users, and the comment author cookies (comment_author_*), which since 4.9.6 are set only if the visitor ticks the "Save my name, email, and website" checkbox on the comment form (Settings > Discussion, option show_comments_cookies_opt_in). Embeds, analytics, fonts loaded from third-party CDNs and plugins add the rest; a block theme that registers its fonts through theme.json or the Font Library serves them locally (see Typography and Fonts).

Tools > Export Personal Data and Tools > Erase Personal Data handle a person’s request by email address:

Core registers exporters for the user’s profile (including the community events location and session tokens), comments and media uploaded by the person, and an eraser for comments. Plugins add their own through the wp_privacy_personal_data_exporters and wp_privacy_personal_data_erasers filters; data that a plugin stores without registering one is not in the export and is not erased.

The screens need export_others_personal_data or erase_others_personal_data; these meta capabilities map to manage_options on a single site (administrators only) and to manage_network on Multisite (super admins only). Erasure additionally requires delete_users.

Core does not ask for cookie consent: it sets no tracking cookies of its own, and the comment cookies already require the opt-in checkbox. Consent banners come from plugins or the site’s own code, and the requirements (prior consent for non-essential cookies, a reject option as easy as accept, a record of consent) come from the law that applies to the site’s readers, such as the EU ePrivacy rules and GDPR.

The WP Consent API (wordpress.org/plugins/wp-consent-api), a community-maintained plugin that is not part of core, gives banners and other plugins a shared vocabulary: a consent management plugin records the visitor’s choice per category (functional, preferences, statistics, statistics-anonymous, marketing), and plugins that set cookies ask wp_has_consent( 'statistics' ) before doing so. The function exists only while the plugin is active, so code that uses it checks function_exists( 'wp_has_consent' ) first. The plugin’s listing names the plugins known to read these signals, Site Kit among them.

For Lumen Press, the order of work is: list every cookie and third-party request the site makes (browser developer tools, with a clean profile), remove what is not needed, self-host fonts, describe the rest on the privacy policy page, and only then choose a banner that blocks or defers the remaining scripts until consent and passes the choice on through consent mode or the WP Consent API.