MarcellusDrum@lemmy.ml to Technology@lemmy.worldEnglish · 28 days agoThis website is for humanslocalghost.devexternal-linkmessage-square62linkfedilinkarrow-up1484arrow-down18cross-posted to: hackernews@lemmy.bestiver.se
arrow-up1476arrow-down1external-linkThis website is for humanslocalghost.devMarcellusDrum@lemmy.ml to Technology@lemmy.worldEnglish · 28 days agomessage-square62linkfedilinkcross-posted to: hackernews@lemmy.bestiver.se
minus-squareaarmea@lemmy.worldlinkfedilinkEnglisharrow-up8·26 days agohttps://localghost.dev/robots.txt User-Agent: * Allow: /
minus-square𝚝𝚛𝚔@aussie.zonelinkfedilinkEnglisharrow-up6·26 days agoIs there a more ignored file on the internet than robots.txt?
https://localghost.dev/robots.txt
User-Agent: * Allow: /Is there a more ignored file on the internet than robots.txt?
Do dnt headers count?