@MarcellusDrum@lemmy.ml to Technology@lemmy.worldEnglish • 7 days agoThis website is for humanslocalghost.devexternal-linkmessage-square61fedilinkarrow-up1476arrow-down19
arrow-up1467arrow-down1external-linkThis website is for humanslocalghost.dev@MarcellusDrum@lemmy.ml to Technology@lemmy.worldEnglish • 7 days agomessage-square61fedilink
minus-square@aarmea@lemmy.worldlinkfedilinkEnglish8•6 days agohttps://localghost.dev/robots.txt User-Agent: * Allow: /
minus-square𝚝𝚛𝚔linkfedilinkEnglish6•5 days agoIs there a more ignored file on the internet than robots.txt?
https://localghost.dev/robots.txt
User-Agent: * Allow: /Is there a more ignored file on the internet than robots.txt?
Do dnt headers count?