Nx2l's Home Lab blog

Still beating my head over squid trying to get a it it work transparently on my network.

Basic setup

image

I have tried several different configs on the squid system but i keep getting the squid access denied page. (so I dont believe anything is wrong on my router (centos/iptables) since squid pages are returned.

Summary
#
# Recommended minimum configuration:
#

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
#acl localnet src 192.168.0.88
acl localnet src 127.0.0.1      # RFC1918 possible internal network
acl localnet src ::1    # RFC1918 possible internal network
#acl localnet src 10.0.0.0/8    # RFC1918 possible internal network
#acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl ilocalnet src 172.16.0.0/12 # RFC1918 possible internal network
#acl localnet src 172.20.0.0/16 # RFC1918 possible internal network
#acl localnet src 172.24.0.0/16 # RFC1918 possible internal network
#acl localnet src 172.28.0.0/16 # RFC1918 possible internal network
#acl localnet src 192.168.0.0/24        # RFC1918 possible internal     network
#acl localnet src fc00::/7       # RFC 4193 local private network range
#acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 80
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports
#http_access allow !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports
#http_access allow CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect         innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS     FROM YOUR CLIENTS
#
# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost
http_access allow ilocalnet
# And finally deny all other access to this proxy
#http_access allow all
http_access deny all

# Squid normally listens to port 3128
http_port 3128 intercept
http_port 3130
# Squid normally listens to port 3128
http_port 3128 intercept
http_port 3130

# Uncomment and adjust the following to add a disk cache directory.
cache_dir ufs /var/spool/squid 50000 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320

image

sample from access.log file

    1559482000.892      0 172.28.3.9 TCP_HIT/200 13081 GET http://spacewalk.jab.us:3129/squid-internal-static/icons/SN.png - HIER_NONE/- image/png
    1559482001.007      0 192.168.0.89 TCP_MISS/403 4322 GET http://192.168.0.1/ - HIER_NONE/- text/html
    1559482001.007      0 172.28.3.9 TCP_MISS/403 4442 GET http://192.168.0.1/ - ORIGINAL_DST/192.168.0.89 text/html
    1559482001.038      0 192.168.0.89 TCP_MISS/403 4256 GET http://192.168.0.1/favicon.ico - HIER_NONE/- text/html
    1559482001.038      0 172.28.3.9 TCP_MISS/403 4376 GET http://192.168.0.1/favicon.ico - ORIGINAL_DST/192.168.0.89 text/html
    1559482001.041      0 172.28.3.9 TCP_HIT/200 13081 GET http://spacewalk.jab.us:3129/squid-internal-static/icons/SN.png - HIER_NONE/- image/png
    1559482003.503      0 172.28.3.9 NONE/000 0 NONE error:transaction-end-before-headers - HIER_NONE/- -
    1559482003.599      0 192.168.0.89 TCP_MISS/403 4385 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - HIER_NONE/- text/html
    1559482003.599      1 172.28.3.9 TCP_MISS/403 4505 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - ORIGINAL_DST/192.168.0.89 text/html
    1559482003.625      0 192.168.0.89 TCP_MISS/403 4199 GET http://www.tldp.org/favicon.ico - HIER_NONE/- text/html
    1559482003.625      0 172.28.3.9 TCP_MISS/403 4319 GET http://www.tldp.org/favicon.ico - ORIGINAL_DST/192.168.0.89 text/html
    1559482003.628      0 172.28.3.9 TCP_HIT/200 13081 GET http://spacewalk.jab.us:3129/squid-internal-static/icons/SN.png - HIER_NONE/- image/png
    1559482004.071      0 192.168.0.89 TCP_MISS/403 4385 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - HIER_NONE/- text/html
    1559482004.071      0 172.28.3.9 TCP_MISS/403 4505 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - ORIGINAL_DST/192.168.0.89 text/html
    1559482004.099      0 192.168.0.89 TCP_MISS/403 4199 GET http://www.tldp.org/favicon.ico - HIER_NONE/- text/html
    1559482004.099      0 172.28.3.9 TCP_MISS/403 4319 GET http://www.tldp.org/favicon.ico - ORIGINAL_DST/192.168.0.89 text/html
    1559482004.103      0 172.28.3.9 TCP_HIT/200 13081 GET http://spacewalk.jab.us:3129/squid-internal-static/icons/SN.png - HIER_NONE/- image/png
    1559482004.405      0 192.168.0.89 TCP_MISS/403 4385 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - HIER_NONE/- text/html
    1559482004.405      0 172.28.3.9 TCP_MISS/403 4505 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - ORIGINAL_DST/192.168.0.89 text/html
    1559482004.416      0 172.28.3.9 TCP_HIT/200 13081 GET http://spacewalk.jab.us:3129/squid-internal-static/icons/SN.png - HIER_NONE/- image/png
    1559482004.423      0 192.168.0.89 TCP_MISS/403 4199 GET http://www.tldp.org/favicon.ico - HIER_NONE/- text/html
    1559482004.423      0 172.28.3.9 TCP_MISS/403 4319 GET http://www.tldp.org/favicon.ico - ORIGINAL_DST/192.168.0.89 text/html
    1559482004.577      0 192.168.0.89 TCP_MISS/403 4385 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - HIER_NONE/- text/html
    1559482004.577      0 172.28.3.9 TCP_MISS/403 4505 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - ORIGINAL_DST/192.168.0.89 text/html
    1559482004.607      0 192.168.0.89 TCP_MISS/403 4199 GET http://www.tldp.org/favicon.ico - HIER_NONE/- text/html
    1559482004.607      0 172.28.3.9 TCP_MISS/403 4319 GET http://www.tldp.org/favicon.ico - ORIGINAL_DST/192.168.0.89 text/html
    1559482004.613      0 172.28.3.9 TCP_HIT/200 13081 GET http://spacewalk.jab.us:3129/squid-internal-static/icons/SN.png - HIER_NONE/- image/png
    1559482005.023      0 192.168.0.89 TCP_MISS/403 4385 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - HIER_NONE/- text/html
    1559482005.023      0 172.28.3.9 TCP_MISS/403 4505 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - ORIGINAL_DST/192.168.0.89 text/html
    1559482005.054      0 192.168.0.89 TCP_MISS/403 4199 GET http://www.tldp.org/favicon.ico - HIER_NONE/- text/html
    1559482005.054      0 172.28.3.9 TCP_MISS/403 4319 GET http://www.tldp.org/favicon.ico - ORIGINAL_DST/192.168.0.89 text/html
    1559482005.057      0 172.28.3.9 TCP_HIT/200 13081 GET http://spacewalk.jab.us:3129/squid-internal-static/icons/SN.png - HIER_NONE/- image/png
    1559482015.430      0 192.168.0.89 TCP_MISS/403 4362 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - HIER_NONE/- text/html
    1559482015.430      0 172.28.3.9 TCP_MISS/403 4482 GET http://www.tldp.org/LDP/Bash-Beginners-Guide/html/f32.html - ORIGINAL_DST/192.168.0.89 text/html
    1559482015.454      0 172.28.3.9 TCP_IMS_HIT/304 323 GET http://spacewalk.jab.us:3129/squid-internal-static/icons/SN.png - HIER_NONE/- image/png

Used some spare parts (plus a couple I had to order) to build a backup for my main storage server.

5 Likes

Your fan/drives config made me chuckle.

3 Likes

as soon as I figure out zfs send/receive … I can kick off the cloning and see how the drive temps look.

1 Like

After playing around…

I just kicked off an rync… when done it should finish in about 12(?) hours… (this is good reason to get 10Gb huh?)

But then I’d need a 10Gb switch… lol

image

Update…

I still havent fixed up the cabling on my “rack” yet… but I dont need that corner of the room (yet) :laughing:

==

If you havent seen my new proxmox hardware “build”

==

I did get the ball rolling on doing some 10Gb stuff…
https://forum.level1techs.com/t/post-what-new-thing-you-acquired-recently/88085/17251?u=nx2l
image

Ive been looking around to see what makes sense with my current setup for adding a couple 10Gb ports to my router/firewall, but options are low since the motherboard is ITX.
(A way around this would be getting a M.2 to pcie X4 adapter so I could add another pcie slot… vs going to mATX motherboard)

Since the aquntia 10GbE nics are quite affordable… (and I already bought one) I guess staying with copper is my plan.
Most dual/quad port options I have found so far X520, X550, X710… etc are mostly $200-$400

I may just figure going mATX or doing the adapter to get another x4 pcie slot is easier to get an Aquantia 10Gb nic added to my router.

Still figuring it out…

Been spending on time on upgrading my proxmox to Ryzen.

U.2 to PCIE adapter will be here soon.

Just updated the BIOS and BMC firmware last night.

Expecting time this weekend to install it

image

image

image

image

1 Like

image image

Im still sorting the networking stuff out… since all the device names are different… but shouldnt be too much longer…

pics coming later…

Photo dump of my work earlier.

Putting this…

Into here…

2 Likes

so I removed the cache device from my zfs pool
(after reviewing the arc summary output… even with noprefetch=0… my l2arc hit% was under 1%)

1 Like

changed from 4Gb HBA to 8Gb HBA…

image

1 Like

just watching a tar backup… of small files…

2 Likes

image

3 Likes

That’s a really neat way to watch the backup take place. What are you using?

.
nmon

1 Like

Do you use pigz to help accelerate your backups?

For example:

#!/bin/bash
# backup /home to zfs pool located at /mnt/tank
tar -cp --use-compress-program=pigz -f /mnt/tank/backups/home/$(date +%F_%R).tar.gz /home;

Makes backups crazy fast because it will use all threads available to run the backup.

CC’ing @oO.o because you might dig this.

3 Likes

Never heard of that before but I’ll check it out… Was just a simple tar backup

Oh it will change your life :wink:

sudo apt install -y pigz

This is what the backups script looks like.

All cores active.

3 Likes

I’ve used pigz before. Very fast. Unfortunate that uncompressing can’t be parallel.