Add header testing scripts for debugging API and security headers
ci/woodpecker/push/woodpecker Pipeline was successful
Details
ci/woodpecker/push/woodpecker Pipeline was successful
Details
This commit is contained in:
parent
7845079307
commit
b950dd2c2a
|
|
@ -0,0 +1,97 @@
|
|||
# API Headers Testing Scripts
|
||||
|
||||
This directory contains scripts for testing the security headers and API compatibility of the Ploughshares application.
|
||||
|
||||
## Scripts Overview
|
||||
|
||||
1. **test_headers.sh** - Tests general HTTP headers for both UI and API endpoints
|
||||
2. **test_api_curl.sh** - Tests API endpoints with various curl commands and parameters
|
||||
3. **test_api_headers.sh** - Specifically analyzes security headers for API compatibility
|
||||
|
||||
## Usage
|
||||
|
||||
All scripts accept an optional URL parameter. If not provided, they default to `http://localhost:5005`.
|
||||
|
||||
```bash
|
||||
# Test against local development environment
|
||||
./test_headers.sh
|
||||
|
||||
# Test against production
|
||||
./test_headers.sh https://ploughshares.nixc.us
|
||||
```
|
||||
|
||||
## Security Headers Analysis
|
||||
|
||||
The scripts check for the following important headers:
|
||||
|
||||
### Content Security Policy (CSP)
|
||||
- Ensures `unsafe-inline` and `unsafe-eval` are present for API compatibility
|
||||
- Verifies `connect-src` includes wildcard (*) for API access
|
||||
- Checks critical protections like `object-src: 'none'` and `frame-ancestors: 'none'`
|
||||
|
||||
### CORS Headers
|
||||
- `Access-Control-Allow-Origin`
|
||||
- `Access-Control-Allow-Methods`
|
||||
- `Access-Control-Allow-Headers`
|
||||
|
||||
### Resource Policies
|
||||
- `Cross-Origin-Resource-Policy` should be `cross-origin` for API endpoints
|
||||
- `Cross-Origin-Resource-Policy` should be `same-origin` for UI endpoints
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If API requests are failing, check the following:
|
||||
|
||||
1. **CSP Issues**: The Content-Security-Policy header must include:
|
||||
- `'unsafe-inline'` and `'unsafe-eval'` in script-src
|
||||
- `*` in connect-src
|
||||
- `data:` and `blob:` in appropriate directives
|
||||
|
||||
2. **CORS Issues**: API endpoints must have:
|
||||
- `Access-Control-Allow-Origin: *` (or specific origins)
|
||||
- `Access-Control-Allow-Methods` with appropriate HTTP methods
|
||||
- `Access-Control-Allow-Headers` with at least `Content-Type` and `Authorization`
|
||||
|
||||
3. **Resource Policy Issues**: API endpoints should have:
|
||||
- `Cross-Origin-Resource-Policy: cross-origin`
|
||||
|
||||
## Current Configuration
|
||||
|
||||
The current configuration uses a permissive CSP that works for both UI and API routes:
|
||||
|
||||
```python
|
||||
csp = {
|
||||
'default-src': ["'self'", "'unsafe-inline'", "'unsafe-eval'", "data:", "blob:"],
|
||||
'script-src': ["'self'", "'unsafe-inline'", "'unsafe-eval'"] + ([f"'sha256-{CSP_JS_HASH}'"] if CSP_JS_HASH else []),
|
||||
'style-src': ["'self'", "'unsafe-inline'"] +
|
||||
([f"'sha256-{CSP_CSS_HASH}'"] if CSP_CSS_HASH else []) +
|
||||
([f"'sha256-{CSP_CUSTOM_CSS_HASH}'"] if CSP_CUSTOM_CSS_HASH else []),
|
||||
'img-src': ["'self'", "data:", "blob:"],
|
||||
'font-src': ["'self'", "data:"],
|
||||
'connect-src': ["'self'", "*"],
|
||||
'manifest-src': "'self'",
|
||||
'object-src': "'none'", # Still explicitly disallow objects
|
||||
'frame-ancestors': "'none'", # Still prevent framing
|
||||
'base-uri': "'self'",
|
||||
'form-action': "'self'"
|
||||
}
|
||||
```
|
||||
|
||||
CORS headers are added for API routes in an after_request handler:
|
||||
|
||||
```python
|
||||
@app.after_request
|
||||
def add_api_headers(response):
|
||||
if request.path.startswith('/api/'):
|
||||
# Add CORS headers for API routes
|
||||
response.headers['Access-Control-Allow-Origin'] = '*'
|
||||
response.headers['Access-Control-Allow-Methods'] = 'GET, POST, PUT, DELETE, OPTIONS'
|
||||
response.headers['Access-Control-Allow-Headers'] = 'Content-Type, Authorization'
|
||||
response.headers['Cross-Origin-Resource-Policy'] = 'cross-origin'
|
||||
else:
|
||||
# For UI routes, add additional security headers
|
||||
for header, value in additional_headers.items():
|
||||
response.headers[header] = value
|
||||
|
||||
return response
|
||||
```
|
||||
|
|
@ -0,0 +1,110 @@
|
|||
#!/bin/bash
|
||||
|
||||
# Colors for better readability
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[0;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Base URLs to test
|
||||
DEV_URL="http://localhost:5005"
|
||||
PROD_URL="https://ploughshares.nixc.us"
|
||||
|
||||
# Default to development URL unless specified
|
||||
URL=${1:-$DEV_URL}
|
||||
|
||||
echo -e "${BLUE}Testing API endpoints with curl for ${URL}${NC}"
|
||||
echo "=================================================="
|
||||
|
||||
# Function to test API endpoint with different curl options
|
||||
test_api() {
|
||||
local endpoint=$1
|
||||
local method=$2
|
||||
local description=$3
|
||||
local headers=$4
|
||||
local data=$5
|
||||
local full_url="${URL}${endpoint}"
|
||||
|
||||
echo -e "\n${YELLOW}Test: ${description}${NC}"
|
||||
echo "------------------------------------------------"
|
||||
|
||||
# Build curl command based on parameters
|
||||
cmd="curl -s"
|
||||
|
||||
# Add method if specified
|
||||
if [ -n "$method" ]; then
|
||||
cmd="$cmd -X $method"
|
||||
fi
|
||||
|
||||
# Add headers if specified
|
||||
if [ -n "$headers" ]; then
|
||||
cmd="$cmd $headers"
|
||||
fi
|
||||
|
||||
# Add data if specified
|
||||
if [ -n "$data" ]; then
|
||||
cmd="$cmd -d '$data'"
|
||||
fi
|
||||
|
||||
# Add URL
|
||||
cmd="$cmd \"$full_url\""
|
||||
|
||||
# Show the command being executed
|
||||
echo -e "${BLUE}Command:${NC} $cmd"
|
||||
|
||||
# Execute the command
|
||||
echo -e "${BLUE}Response:${NC}"
|
||||
eval $cmd | jq 2>/dev/null || echo "Response is not valid JSON"
|
||||
|
||||
echo "------------------------------------------------"
|
||||
}
|
||||
|
||||
# Test 1: Basic GET request to API test endpoint
|
||||
test_api "/api/test" "GET" "Basic GET request to API test endpoint" "" ""
|
||||
|
||||
# Test 2: GET request with Accept header
|
||||
test_api "/api/test" "GET" "GET request with Accept: application/json header" "-H \"Accept: application/json\"" ""
|
||||
|
||||
# Test 3: OPTIONS request (CORS preflight)
|
||||
test_api "/api/test" "OPTIONS" "OPTIONS request (CORS preflight)" "-H \"Origin: http://example.com\"" ""
|
||||
|
||||
# Test 4: POST request with JSON data
|
||||
test_api "/api/transactions" "POST" "POST request with JSON data" "-H \"Content-Type: application/json\"" "{\"test\": \"data\"}"
|
||||
|
||||
# Test 5: GET request with custom User-Agent
|
||||
test_api "/api/test" "GET" "GET request with custom User-Agent" "-H \"User-Agent: CustomClient/1.0\"" ""
|
||||
|
||||
# Test 6: GET request with Authorization header
|
||||
test_api "/api/test" "GET" "GET request with Authorization header" "-H \"Authorization: Bearer test-token\"" ""
|
||||
|
||||
# Test 7: POST request with form data
|
||||
test_api "/api/transactions" "POST" "POST request with form data" "-H \"Content-Type: application/x-www-form-urlencoded\"" "test=data"
|
||||
|
||||
# Test 8: GET request with multiple custom headers
|
||||
test_api "/api/test" "GET" "GET request with multiple custom headers" "-H \"Accept: application/json\" -H \"X-Custom-Header: test\"" ""
|
||||
|
||||
echo -e "\n${BLUE}API curl tests completed for ${URL}${NC}"
|
||||
|
||||
# Now test the API with a more complex curl command that simulates a real client
|
||||
echo -e "\n${YELLOW}Simulating a real API client with complex curl command${NC}"
|
||||
echo "------------------------------------------------"
|
||||
|
||||
echo -e "${BLUE}Command:${NC}"
|
||||
echo "curl -v -X POST \"${URL}/api/transactions\" \\"
|
||||
echo " -H \"Content-Type: application/json\" \\"
|
||||
echo " -H \"Accept: application/json\" \\"
|
||||
echo " -H \"User-Agent: APIClient/1.0\" \\"
|
||||
echo " -H \"Authorization: Bearer test-token\" \\"
|
||||
echo " -d '{\"transaction_type\":\"test\",\"amount\":100,\"recipient\":\"Test Corp\"}'"
|
||||
|
||||
echo -e "\n${BLUE}Response:${NC}"
|
||||
curl -v -X POST "${URL}/api/transactions" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Accept: application/json" \
|
||||
-H "User-Agent: APIClient/1.0" \
|
||||
-H "Authorization: Bearer test-token" \
|
||||
-d '{"transaction_type":"test","amount":100,"recipient":"Test Corp"}' 2>&1
|
||||
|
||||
echo -e "\n------------------------------------------------"
|
||||
echo -e "${GREEN}All tests completed.${NC}"
|
||||
|
|
@ -0,0 +1,138 @@
|
|||
#!/bin/bash
|
||||
|
||||
# Colors for better readability
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[0;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Base URLs to test
|
||||
DEV_URL="http://localhost:5005"
|
||||
PROD_URL="https://ploughshares.nixc.us"
|
||||
|
||||
# Default to development URL unless specified
|
||||
URL=${1:-$DEV_URL}
|
||||
|
||||
echo -e "${BLUE}Testing API Security Headers for ${URL}${NC}"
|
||||
echo "=================================================="
|
||||
|
||||
# Function to test an endpoint's headers
|
||||
test_headers() {
|
||||
local endpoint=$1
|
||||
local description=$2
|
||||
local full_url="${URL}${endpoint}"
|
||||
|
||||
echo -e "\n${YELLOW}Testing headers for ${description} (${endpoint})${NC}"
|
||||
echo "------------------------------------------------"
|
||||
|
||||
# Get headers only with curl
|
||||
headers=$(curl -s -I "${full_url}")
|
||||
|
||||
# Display all headers for reference
|
||||
echo -e "${BLUE}All Headers:${NC}"
|
||||
echo "$headers" | grep -v "Date:" | grep -v "Server:"
|
||||
|
||||
echo -e "\n${BLUE}Security Header Analysis:${NC}"
|
||||
|
||||
# Check for Content-Security-Policy
|
||||
if echo "$headers" | grep -q "Content-Security-Policy"; then
|
||||
csp=$(echo "$headers" | grep "Content-Security-Policy" | sed "s/Content-Security-Policy: //")
|
||||
echo -e "${GREEN}✓ Content-Security-Policy:${NC}"
|
||||
|
||||
# Check for unsafe-inline and unsafe-eval in script-src
|
||||
if echo "$csp" | grep -q "script-src.*'unsafe-inline'"; then
|
||||
echo -e " ${YELLOW}⚠ script-src contains 'unsafe-inline' - needed for API clients${NC}"
|
||||
else
|
||||
echo -e " ${RED}✗ script-src missing 'unsafe-inline' - may cause issues with some clients${NC}"
|
||||
fi
|
||||
|
||||
if echo "$csp" | grep -q "script-src.*'unsafe-eval'"; then
|
||||
echo -e " ${YELLOW}⚠ script-src contains 'unsafe-eval' - needed for API clients${NC}"
|
||||
else
|
||||
echo -e " ${RED}✗ script-src missing 'unsafe-eval' - may cause issues with some clients${NC}"
|
||||
fi
|
||||
|
||||
# Check for connect-src
|
||||
if echo "$csp" | grep -q "connect-src.*\\*"; then
|
||||
echo -e " ${GREEN}✓ connect-src includes * - good for API access${NC}"
|
||||
else
|
||||
echo -e " ${RED}✗ connect-src restrictive - may block API access${NC}"
|
||||
fi
|
||||
else
|
||||
echo -e "${RED}✗ Content-Security-Policy header not found${NC}"
|
||||
fi
|
||||
|
||||
# Check for CORS headers for API endpoints
|
||||
if [[ "$endpoint" == "/api/"* ]]; then
|
||||
echo -e "\n${BLUE}CORS Headers (critical for API):${NC}"
|
||||
|
||||
if echo "$headers" | grep -q "Access-Control-Allow-Origin"; then
|
||||
origin=$(echo "$headers" | grep "Access-Control-Allow-Origin" | sed "s/Access-Control-Allow-Origin: //")
|
||||
echo -e "${GREEN}✓ Access-Control-Allow-Origin: $origin${NC}"
|
||||
else
|
||||
echo -e "${RED}✗ Access-Control-Allow-Origin header not found - CORS will fail${NC}"
|
||||
fi
|
||||
|
||||
if echo "$headers" | grep -q "Access-Control-Allow-Methods"; then
|
||||
methods=$(echo "$headers" | grep "Access-Control-Allow-Methods" | sed "s/Access-Control-Allow-Methods: //")
|
||||
echo -e "${GREEN}✓ Access-Control-Allow-Methods: $methods${NC}"
|
||||
else
|
||||
echo -e "${RED}✗ Access-Control-Allow-Methods header not found - CORS will fail${NC}"
|
||||
fi
|
||||
|
||||
if echo "$headers" | grep -q "Access-Control-Allow-Headers"; then
|
||||
allowed_headers=$(echo "$headers" | grep "Access-Control-Allow-Headers" | sed "s/Access-Control-Allow-Headers: //")
|
||||
echo -e "${GREEN}✓ Access-Control-Allow-Headers: $allowed_headers${NC}"
|
||||
else
|
||||
echo -e "${RED}✗ Access-Control-Allow-Headers header not found - CORS will fail${NC}"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check for Cross-Origin-Resource-Policy
|
||||
if echo "$headers" | grep -q "Cross-Origin-Resource-Policy"; then
|
||||
corp=$(echo "$headers" | grep "Cross-Origin-Resource-Policy" | sed "s/Cross-Origin-Resource-Policy: //")
|
||||
echo -e "\n${BLUE}Resource Policy:${NC}"
|
||||
|
||||
if [[ "$endpoint" == "/api/"* ]]; then
|
||||
if [[ "$corp" == "cross-origin" ]]; then
|
||||
echo -e "${GREEN}✓ Cross-Origin-Resource-Policy: $corp - correct for API${NC}"
|
||||
else
|
||||
echo -e "${RED}✗ Cross-Origin-Resource-Policy: $corp - should be 'cross-origin' for API endpoints${NC}"
|
||||
fi
|
||||
else
|
||||
if [[ "$corp" == "same-origin" ]]; then
|
||||
echo -e "${GREEN}✓ Cross-Origin-Resource-Policy: $corp - correct for UI${NC}"
|
||||
else
|
||||
echo -e "${YELLOW}⚠ Cross-Origin-Resource-Policy: $corp - should be 'same-origin' for UI endpoints${NC}"
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo -e "${YELLOW}⚠ Cross-Origin-Resource-Policy header not found${NC}"
|
||||
fi
|
||||
|
||||
echo "------------------------------------------------"
|
||||
}
|
||||
|
||||
# Test UI endpoint
|
||||
test_headers "/" "Main UI page"
|
||||
|
||||
# Test API endpoints
|
||||
test_headers "/api/test" "API test endpoint"
|
||||
|
||||
# Test OPTIONS request for CORS preflight
|
||||
echo -e "\n${YELLOW}Testing OPTIONS request for CORS preflight${NC}"
|
||||
echo "------------------------------------------------"
|
||||
curl -s -X OPTIONS -H "Origin: http://example.com" -I "${URL}/api/test" | grep -v "Date:" | grep -v "Server:"
|
||||
echo "------------------------------------------------"
|
||||
|
||||
# Test with a real-world API client simulation
|
||||
echo -e "\n${YELLOW}Testing with a simulated API client${NC}"
|
||||
echo "------------------------------------------------"
|
||||
echo -e "${BLUE}Command:${NC} curl -s -X GET -H \"Origin: http://example.com\" -H \"Accept: application/json\" \"${URL}/api/test\""
|
||||
response=$(curl -s -X GET -H "Origin: http://example.com" -H "Accept: application/json" "${URL}/api/test")
|
||||
echo -e "${BLUE}Response:${NC}"
|
||||
echo "$response" | jq 2>/dev/null || echo "$response"
|
||||
echo "------------------------------------------------"
|
||||
|
||||
echo -e "\n${GREEN}All header tests completed.${NC}"
|
||||
|
|
@ -0,0 +1,123 @@
|
|||
#!/bin/bash
|
||||
|
||||
# Colors for better readability
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[0;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Base URLs to test
|
||||
DEV_URL="http://localhost:5005"
|
||||
PROD_URL="https://ploughshares.nixc.us"
|
||||
|
||||
# Default to development URL unless specified
|
||||
URL=${1:-$DEV_URL}
|
||||
|
||||
echo -e "${BLUE}Testing headers for ${URL}${NC}"
|
||||
echo "=================================================="
|
||||
|
||||
# Function to test an endpoint
|
||||
test_endpoint() {
|
||||
local endpoint=$1
|
||||
local method=$2
|
||||
local description=$3
|
||||
local full_url="${URL}${endpoint}"
|
||||
|
||||
echo -e "\n${YELLOW}Testing ${method} ${endpoint}: ${description}${NC}"
|
||||
echo "------------------------------------------------"
|
||||
|
||||
if [ "$method" == "HEAD" ]; then
|
||||
# Use -I for HEAD requests
|
||||
curl -s -I "${full_url}" | grep -v "Date:" | grep -v "Server:"
|
||||
elif [ "$method" == "OPTIONS" ]; then
|
||||
# Use -X OPTIONS for OPTIONS requests
|
||||
curl -s -X OPTIONS "${full_url}" -I | grep -v "Date:" | grep -v "Server:"
|
||||
elif [ "$method" == "GET" ]; then
|
||||
# First check headers
|
||||
echo -e "${BLUE}Headers:${NC}"
|
||||
curl -s -I "${full_url}" | grep -v "Date:" | grep -v "Server:"
|
||||
|
||||
# Then check response body
|
||||
echo -e "\n${BLUE}Response:${NC}"
|
||||
curl -s "${full_url}"
|
||||
echo -e "\n"
|
||||
else
|
||||
echo "Unsupported method: ${method}"
|
||||
fi
|
||||
|
||||
echo "------------------------------------------------"
|
||||
}
|
||||
|
||||
# Function to check for specific security headers
|
||||
check_security_headers() {
|
||||
local endpoint=$1
|
||||
local full_url="${URL}${endpoint}"
|
||||
|
||||
echo -e "\n${YELLOW}Checking security headers for ${endpoint}${NC}"
|
||||
echo "------------------------------------------------"
|
||||
|
||||
# Store headers in a variable
|
||||
headers=$(curl -s -I "${full_url}")
|
||||
|
||||
# Check for important security headers
|
||||
check_header "Content-Security-Policy" "$headers"
|
||||
check_header "Cross-Origin-Resource-Policy" "$headers"
|
||||
check_header "Cross-Origin-Embedder-Policy" "$headers"
|
||||
check_header "Cross-Origin-Opener-Policy" "$headers"
|
||||
check_header "X-Frame-Options" "$headers"
|
||||
check_header "X-Content-Type-Options" "$headers"
|
||||
check_header "X-XSS-Protection" "$headers"
|
||||
check_header "Referrer-Policy" "$headers"
|
||||
|
||||
# For API endpoints, check CORS headers
|
||||
if [[ "$endpoint" == "/api/"* ]]; then
|
||||
check_header "Access-Control-Allow-Origin" "$headers"
|
||||
check_header "Access-Control-Allow-Methods" "$headers"
|
||||
check_header "Access-Control-Allow-Headers" "$headers"
|
||||
fi
|
||||
|
||||
echo "------------------------------------------------"
|
||||
}
|
||||
|
||||
# Function to check for a specific header
|
||||
check_header() {
|
||||
local header=$1
|
||||
local headers=$2
|
||||
|
||||
if echo "$headers" | grep -q "$header"; then
|
||||
header_value=$(echo "$headers" | grep "$header" | sed "s/$header: //")
|
||||
echo -e "${GREEN}✓ $header: $header_value${NC}"
|
||||
else
|
||||
echo -e "${RED}✗ $header not found${NC}"
|
||||
fi
|
||||
}
|
||||
|
||||
# Test UI endpoints
|
||||
test_endpoint "/" "HEAD" "Main page headers"
|
||||
check_security_headers "/"
|
||||
|
||||
# Test API endpoints
|
||||
test_endpoint "/api/test" "HEAD" "API test endpoint headers"
|
||||
check_security_headers "/api/test"
|
||||
|
||||
test_endpoint "/api/test" "GET" "API test endpoint response"
|
||||
|
||||
# Test OPTIONS request for CORS preflight
|
||||
test_endpoint "/api/test" "OPTIONS" "API CORS preflight request"
|
||||
|
||||
# Test a POST request to API
|
||||
echo -e "\n${YELLOW}Testing POST to /api/transactions${NC}"
|
||||
echo "------------------------------------------------"
|
||||
curl -s -X POST "${URL}/api/transactions" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"test": "data"}' | jq 2>/dev/null || echo "Response is not valid JSON"
|
||||
echo -e "\n------------------------------------------------"
|
||||
|
||||
# Test with different Accept headers
|
||||
echo -e "\n${YELLOW}Testing with Accept: application/json header${NC}"
|
||||
echo "------------------------------------------------"
|
||||
curl -s -H "Accept: application/json" "${URL}/api/test" | jq 2>/dev/null || echo "Response is not valid JSON"
|
||||
echo -e "\n------------------------------------------------"
|
||||
|
||||
echo -e "\n${BLUE}Tests completed for ${URL}${NC}"
|
||||
Loading…
Reference in New Issue