we recently moved from a flat structure to a deeper hierarchy to better organize our product categories. since the migration, i have noticed that some deep-level pages are taking much longer to appear in the index than they used to. it seems like the crawler is hitting a wall once it gets past the third level of the directory. i checked the logs and everything looks fine regarding status codes, but the crawl depth is clearly an issue.
current setupthe site uses a heavy amount of javascript to render the category trees which might be the culprit here. we are using
window.location.href
for some of our navigation links instead of standard anchors. i am worried that the bot is not following the dynamic paths correctly.
>the budget is being wasted on high-level pages while the bottom of the funnel stays invisiblei have tried adding more internal links from the homepage to these deeper nodes, but it hasn't changed much. does anyone have experience with
improving crawl depth for heavily nested sites without bloating the link equity? i am also considering moving some of this logic to server-side rendering
but that would be a massive engineering headache . has anyone else seen
significant delays in indexing when using dynamic navigation like this?