Understand the idea

A crawler needs discoverable URLs and content it can retrieve. Providing the complete article in the initial HTML reduces dependence on script execution.

Use ordinary anchors with href attributes for internal links. Give each page a descriptive title and a stable canonical URL. A sitemap can help discovery, but does not guarantee indexing. Robots rules manage crawling, while noindex addresses indexing for crawlers that support and can read it.

Read the example

These are two fragments: metadata belongs in head; the visible link belongs in body. Replace the example domain and provide the linked page.

HTML · EXAMPLE
<!-- Inside head -->
<title>Links and paths | My web guide</title>
<meta name="description" content="Understand relative links and fix broken file paths.">
<link rel="canonical" href="https://example.com/guides/links/">

<!-- Inside body -->
<a href="/guides/images/">Images and alternative text</a>

A small mistake, explained

What goes wrong

An allow rule in robots.txt does not override a hosting-level challenge or guarantee that a search engine chooses to index a page.

How to fix it. Check the returned HTML and HTTP status, then review the actual hosting rules. Keep private content protected with access control, not robots.txt.

Try it yourself

View the page source with JavaScript disabled. Confirm that the article and its navigation links are still present.

Further reading

Google Search Central — SEO Starter Guide

Original explanation and example prepared for HTML code FYI with AI assistance. Test the code in your own context. How these guides are made.