Cynicus Rex@lemmy.ml to Privacy@lemmy.mlEnglish · 1 year agoHow to block AI Crawler Bots using robots.txt filewww.cyberciti.bizexternal-linkmessage-square60linkfedilinkarrow-up159arrow-down111
arrow-up148arrow-down1external-linkHow to block AI Crawler Bots using robots.txt filewww.cyberciti.bizCynicus Rex@lemmy.ml to Privacy@lemmy.mlEnglish · 1 year agomessage-square60linkfedilink
minus-squareJackbyDev@programming.devlinkfedilinkEnglisharrow-up3arrow-down3·1 year agoIs it a lie or a simplification for beginners?
minus-squarethanks_shakey_snake@lemmy.calinkfedilinkarrow-up13·1 year agoLie. Or at best, dangerously wrong. Like saying “Crosswalks make cars incapable of harming pedestrians who stay within them.”
minus-squareJackbyDev@programming.devlinkfedilinkEnglisharrow-up1arrow-down5·1 year agoIt’s better than saying something like “there’s no point in robots.txt because bots can disobey is” though.
minus-squarethanks_shakey_snake@lemmy.calinkfedilinkarrow-up3·1 year agoMaybe? But it’s not like that’s the only alternative thing to say, lol
minus-squareReversalHatchery@beehaw.orglinkfedilinkEnglisharrow-up2arrow-down1·edit-21 year agoIs it, though? I mean, robots.txt is the Do Not Track of the opposite side of the connection.
minus-squaremox@lemmy.sdf.orglinkfedilinkarrow-up2·1 year agoAssuring someone that they have control of something and the safety that comes with it, when in fact they do not, is well outside the realm of a simplification. It’s just plain false. It can even be dangerous.
minus-squareEager Eagle@lemmy.worldlinkfedilinkEnglisharrow-up1·1 year agothe word disallow is right there
Is it a lie or a simplification for beginners?
Lie. Or at best, dangerously wrong. Like saying “Crosswalks make cars incapable of harming pedestrians who stay within them.”
It’s better than saying something like “there’s no point in robots.txt because bots can disobey is” though.
Maybe? But it’s not like that’s the only alternative thing to say, lol
Is it, though?
I mean, robots.txt is the Do Not Track of the opposite side of the connection.
Assuring someone that they have control of something and the safety that comes with it, when in fact they do not, is well outside the realm of a simplification. It’s just plain false. It can even be dangerous.
the word disallow is right there