Pipe Network (Testnet) - Monitor

📊 Monitoring Your Pipe Network Testnet Node

After your Pipe node is running via systemd, here's how to monitor its performance, maintain optimal operation, and troubleshoot common issues.

---

✅ Check Service Status

# Check if service is active and running
sudo systemctl status popcache

# Quick service overview
sudo systemctl is-active popcache
sudo systemctl is-enabled popcache

# Check service start time and uptime
systemctl show popcache --property=ActiveEnterTimestamp

🔁 View Real-time Logs

# Follow logs in real-time
sudo journalctl -u popcache -f

# View last 100 log entries
sudo journalctl -u popcache -n 100 --no-pager

# Filter logs by time period
sudo journalctl -u popcache --since "1 hour ago"
sudo journalctl -u popcache --since "2025-01-20 10:00:00"

# Search for specific keywords
sudo journalctl -u popcache | grep -i "error\|warn\|cache\|peer"

📈 Check Node Performance

# Check built-in health endpoint
curl http://localhost:80/health

# Get node metrics
curl http://localhost:80/metrics

# Check node information
curl http://localhost:80/info

# Test HTTPS endpoint (may show cert warning)
curl -k https://localhost:443/health

🔍 Monitor Cache Performance

# Check cache statistics
curl http://localhost:80/metrics | grep -i cache

# Monitor cache hit/miss ratio
curl http://localhost:80/metrics | grep -E "(cache_hits|cache_misses)"

# Check disk cache usage
du -sh /opt/popcache/cache

# Monitor memory usage
free -h
cat /proc/meminfo | grep -E "(MemTotal|MemAvailable|Cached)"

---

🔄 Maintenance Operations

🚀 Restart Service (Safe Method)

# Graceful restart
sudo systemctl restart popcache

# Check status after restart
sudo systemctl status popcache

# View logs after restart
sudo journalctl -u popcache -n 50 --no-pager

📦 Update Node Binary

# Stop the service
sudo systemctl stop popcache

# Switch to popcache user
sudo su - popcache

# Backup current binary
cd /opt/popcache
cp pop pop.backup

# Download latest version (check for updates)
wget -O pop "https://github.com/PipeNetwork/pip-node/releases/latest/download/pop-linux-amd64"
chmod +x pop

# Test new binary
./pop --version
./pop --help

# Exit popcache user
exit

# Start service with new binary
sudo systemctl start popcache

# Verify update
sudo journalctl -u popcache -n 20 --no-pager

🧹 Clean Cache Data (if needed)

# Stop service
sudo systemctl stop popcache

# Clear cache directory (⚠️ This will remove all cached content)
sudo rm -rf /opt/popcache/cache/*

# Restart service
sudo systemctl start popcache

# Monitor cache rebuilding
sudo journalctl -u popcache -f

💾 Backup Configuration

# Create backup directory
mkdir -p ~/pipe-backups/$(date +%Y%m%d)

# Backup configuration
sudo cp /opt/popcache/config.json ~/pipe-backups/$(date +%Y%m%d)/
sudo cp /etc/systemd/system/popcache.service ~/pipe-backups/$(date +%Y%m%d)/

# Backup system configuration
cp /etc/sysctl.d/99-popcache.conf ~/pipe-backups/$(date +%Y%m%d)/ 2>/dev/null || true
cp /etc/security/limits.d/popcache.conf ~/pipe-backups/$(date +%Y%m%d)/ 2>/dev/null || true

# Create archive
tar -czf ~/pipe-backups/pipe-config-$(date +%Y%m%d).tar.gz ~/pipe-backups/$(date +%Y%m%d)/

---

📱 Performance Monitoring

🖥️ System Resource Monitoring

# Monitor real-time system resources
htop

# Check CPU usage by popcache process
top -p $(pgrep -f "/opt/popcache/pop")

# Monitor disk I/O
iostat -x 1

# Check network connections
ss -tuln | grep -E "(80|443)"
netstat -tulnp | grep -E "(80|443)"

📊 Network Performance

# Test external connectivity
curl -I http://$(curl -s ifconfig.me):80
curl -k -I https://$(curl -s ifconfig.me):443

# Monitor network traffic
iftop -i eth0

# Check connection counts
ss -s

# Monitor bandwidth usage
vnstat -i eth0

📈 Create Monitoring Dashboard Script

# Create monitoring script
nano ~/pipe_dashboard.sh

Add this content:

#!/bin/bash
# Pipe Network Node Dashboard

clear
echo "==================================="
echo "    PIPE NETWORK NODE DASHBOARD"
echo "==================================="
echo ""

# Service Status
echo "🟢 SERVICE STATUS:"
sudo systemctl is-active popcache >/dev/null 2>&1 && echo "✅ Service: ACTIVE" || echo "❌ Service: INACTIVE"
echo ""

# Cache Statistics
echo "📊 CACHE STATISTICS:"
CACHE_SIZE=$(du -sh /opt/popcache/cache 2>/dev/null | cut -f1)
echo "Cache Size: $CACHE_SIZE"

# System Resources
echo ""
echo "🖥️  SYSTEM RESOURCES:"
echo "CPU Usage: $(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | sed 's/%us,//')%"
echo "Memory: $(free -h | awk 'NR==2{printf "%.1f%% (%s/%s)", $3*100/$2, $3, $2}')"
echo "Disk: $(df -h /opt/popcache | awk 'NR==2{printf "%s/%s (%s)", $3, $2, $5}')"

# Network Info
echo ""
echo "🌐 NETWORK INFO:"
echo "External IP: $(curl -s ifconfig.me 2>/dev/null || echo "Unable to fetch")"
echo "HTTP Port: $(ss -tuln | grep ':80 ' >/dev/null && echo "✅ Open" || echo "❌ Closed")"
echo "HTTPS Port: $(ss -tuln | grep ':443 ' >/dev/null && echo "✅ Open" || echo "❌ Closed")"

# Recent Logs
echo ""
echo "📝 RECENT ACTIVITY:"
sudo journalctl -u popcache -n 5 --no-pager --output=short-iso
echo ""
echo "==================================="

# Make executable
chmod +x ~/pipe_dashboard.sh

# Run dashboard
~/pipe_dashboard.sh

⏰ Automated Health Checks

# Create health check script
nano ~/pipe_health_check.sh

Add this content:

#!/bin/bash
# Pipe Network Health Check Script

LOG_FILE="/opt/popcache/logs/health_check.log"
DATE=$(date '+%Y-%m-%d %H:%M:%S')

# Check if service is running
if ! systemctl is-active --quiet popcache; then
    echo "[$DATE] ERROR: popcache service is not running" >> $LOG_FILE
    sudo systemctl restart popcache
    sleep 30
    if ! systemctl is-active --quiet popcache; then
        echo "[$DATE] CRITICAL: Failed to restart popcache service" >> $LOG_FILE
        exit 1
    fi
    echo "[$DATE] INFO: popcache service restarted successfully" >> $LOG_FILE
fi

# Check HTTP endpoint
if ! curl -f -s http://localhost:80/health >/dev/null 2>&1; then
    echo "[$DATE] WARNING: HTTP health check failed" >> $LOG_FILE
    # Give it another chance
    sleep 10
    if ! curl -f -s http://localhost:80/health >/dev/null 2>&1; then
        echo "[$DATE] ERROR: HTTP endpoint not responding" >> $LOG_FILE
        sudo systemctl restart popcache
        exit 1
    fi
fi

# Check disk space (warn if >90% full)
DISK_USAGE=$(df /opt/popcache | tail -1 | awk '{print $5}' | sed 's/%//')
if [[ $DISK_USAGE -gt 90 ]]; then
    echo "[$DATE] WARNING: Disk usage high: ${DISK_USAGE}%" >> $LOG_FILE
fi

# Check memory usage (warn if >90% used)
MEM_USAGE=$(free | grep Mem | awk '{printf("%.0f", $3/$2 * 100.0)}')
if [[ $MEM_USAGE -gt 90 ]]; then
    echo "[$DATE] WARNING: Memory usage high: ${MEM_USAGE}%" >> $LOG_FILE
fi

echo "[$DATE] INFO: Health check completed successfully" >> $LOG_FILE
exit 0

# Make executable
chmod +x ~/pipe_health_check.sh

# Add to crontab (runs every 5 minutes)
(crontab -l 2>/dev/null; echo "*/5 * * * * $HOME/pipe_health_check.sh") | crontab -

# View crontab
crontab -l

---

🔧 Troubleshooting & Optimization

🚨 Common Issues & Solutions

# Issue: Service won't start
# Check configuration
sudo su - popcache
cd /opt/popcache
./pop --check-config  # If this command exists

# Check permissions
ls -la /opt/popcache/
sudo chown -R popcache:popcache /opt/popcache

# Issue: Port binding failed
# Check what's using the ports
sudo lsof -i :80
sudo lsof -i :443

# Stop conflicting services
sudo systemctl stop apache2 nginx

# Issue: High memory usage
# Restart service to clear memory
sudo systemctl restart popcache

# Monitor memory after restart
watch -n 5 'free -h'

📋 Performance Optimization

# Monitor cache hit rate
curl http://localhost:80/metrics | grep -E "(hit|miss)" | head -10

# Optimize cache configuration (edit config.json)
sudo nano /opt/popcache/config.json

# Restart after configuration changes
sudo systemctl restart popcache

📅 Daily Maintenance Checklist

# 1. Check service status
sudo systemctl status popcache

# 2. Review logs for errors
sudo journalctl -u popcache --since "24 hours ago" | grep -i error

# 3. Check disk space
df -h /opt/popcache

# 4. Monitor cache size
du -sh /opt/popcache/cache

# 5. Test endpoints
curl http://localhost:80/health
curl http://localhost:80/metrics

# 6. Check external connectivity
curl -I http://$(curl -s ifconfig.me):80

---

🌐 Network Participation & Metrics

📊 Monitor Network Participation

# Get node metrics
curl http://localhost:80/metrics

# Check node info
curl http://localhost:80/info | jq

# Monitor peer connections (if available)
curl http://localhost:80/peers 2>/dev/null | jq || echo "Peers endpoint not available"

🎯 Performance Targets

• Uptime: Aim for 99%+ service availability
• Response Time: HTTP endpoints should respond within 100ms
• Cache Hit Rate: Target 70%+ cache hit ratio for optimal performance
• Disk Usage: Keep cache disk usage below 90%
• Memory Usage: Monitor and restart if memory grows beyond configured limits

---

🎉 Success Indicators

✅ Service running continuously
✅ **