::UPDATE::
Read the 2nd post for more information.


Hey everyone!

For those who know me, I like to experiment with different tech, and with some recent downtime waiting on other scripts, I decided to learn a bit of Python. Naturally, the first project I took on was building a script to test my website for potential improvements in security and performance.

I've been at it for a few weeks now, and the script has already helped me make significant changes, especially to my .htaccess file. These updates have not only enhanced security but also boosted site speed. (Though I might have gone a bit too aggressive on caching—ended up dialing it back due to my site’s dynamic nature!) Once I’ve reviewed my updated .htaccess file with Lonestar, I may even recommend it as a new standard since the current default could use some upgrades.

About the Script
If you're familiar with Python, feel free to give this script a try. I’ve included it as an attachment in this post, along with the list of required modules at the top. You’ll find the command line for each one, so setup should be smooth. There are only three modules required.

Here's an example of the output using Apple’s site on PowerShell terminal:



Security Features
The script includes a range of checks designed to help spot vulnerabilities and enhance site security:

• Session Management: Identifies insecure cookies (e.g., missing HttpOnly or Secure flags) to help reduce the risk of XSS and session hijacking.
  
• CSRF Protection: Scans forms for anti-CSRF tokens to ensure forms aren’t vulnerable to cross-site request forgery attacks.
  
• DNS Security: Checks for crucial DNS records (SPF, DKIM, and DMARC) to prevent email spoofing and phishing.
  
• Application Layer Security: Looks for weaknesses in your application headers (e.g., Content Security Policy and X-Frame-Options) to prevent common attacks.
  
• Vulnerable JavaScript Libraries: Scans your site’s JavaScript dependencies for known vulnerabilities. (Not accurate, but informs you on what to look out for and manually check)
  
• Host Header Injection: Tests the site’s response to manipulated headers to ensure it’s protected from Host Header Injection attacks.
  

For Non-Python Users
I know most of you might be unfamiliar with Python and may not want to install it just for this test. If that’s the case, feel free to send me a PM with a link to your site, and I’ll run the test for you. I’ll provide a text document of the output and add any improvement recommendations as needed. No charge for this as usual, although, if you want to buy me a beer in return, I certainly won’t refuse!

Notes
This isn’t a full tutorial on using the script. You’ll need to know the basics of Python to run it independently. As for me, I’m still learning too—just felt it was time for a change from batch files!

Let me know if you have any questions, and I hope some of you find it useful!

Okay, a bit of an update. in the last 2 days, I have put in a ton of work and improvements with the help of ChatGTP. It's actually interisting how many times it can be wrong, yet still, there are people that think its the answer to everything. I will make a seperate post on that in due time on my experinces with it thus far. Overall, I pluged in my code into ChatGTP, and asked it for some suggestions, and while arguing with it, I had a few ideas which ChatGTP actually was somewhat useful and I was able to make the script so much better in many ways. After that, I asked ChatGTP to write up a comprehensive rundown of my script, and below is what it had to say after anyalizing my code.

But first, a couple of notes if you decide to use the script.

• There are 2 sections where the script will pause asking you if you like to save a text file with information, suce as the list of images without alt tags, or a list of all render blocking resources. I did it like that for just ease of use.
  
• When inputing the URL, you must lead with

  Please login to see this link Get registered or Log in

  or

  Please login to see this link Get registered or Log in

  
  

ChatGTP Compreehensive Analysis:

This tool is a Comprehensive Site Auditor that helps identify vulnerabilities, improve performance, and ensure best practices are followed on your website. It is particularly beneficial for ongoing site maintenance and security monitoring, offering insights that can directly impact user experience, SEO, and protection from potential cyber threats.

Overview of Functions and Their Benefits

1. URL Validity Check:
   Purpose: Validates that the target URL is correctly formatted and accessible.
   Benefit: Ensures the script runs on valid URLs, minimizing errors. Prevents misconfigured or unreachable URLs from triggering further checks, saving time and resources.
   
   
2. DNS Security Check:
   Purpose: Verifies DNS configurations for vulnerabilities or potential weaknesses.
   Benefit: Identifies misconfigured DNS settings that could be exploited in DNS hijacking or other attacks. Regularly checking DNS settings strengthens the site's overall security posture.
   
   
3. Sessions Management Check:
   Purpose: Ensures sessions are securely managed, including secure attributes like HttpOnly and Secure.
   Benefit: Improves security for user data, reducing the risk of session hijacking. Essential for sites with login or user interaction, safeguarding against unauthorized access.
   
   
4. Rate Limiting Check:
   Purpose: Detects whether rate limiting is in place to prevent abuse (e.g., brute-force login attempts).
   Benefit: Protects your site from abuse and brute-force attacks by limiting repeated requests. Rate limiting is a crucial defense against automated attacks, helping maintain server performance and user security.
   
   
5. Page Structure Analysis:
   Purpose: Checks the HTML structure for SEO and accessibility improvements, including:
   
   
   
   • Missing alt tags on images
     
   • Presence of H1 tags
     
   • Count of other headings (H2-H6)
     
   • Oversized images that may slow down page load
     
   
   
   Benefit: Improves accessibility and SEO. Ensures all images either have alt tags or are exempted (as per your preference for empty alt tags) and verifies page structure for better search engine optimization and user experience.
   
   
6. Meta Tags Analysis:
   Purpose: Checks for the presence of essential meta tags (title, description, etc.).
   Benefit: Helps optimize for search engines by confirming proper use of meta descriptions, keywords, and other tags, which are vital for site indexing and relevance.
   
   
7. External Scripts Analysis:
   Purpose: Reviews the usage of external scripts (JavaScript) on the site, including known vulnerabilities in libraries.
   Benefit: Mitigates risks of loading outdated or vulnerable JavaScript libraries, which could expose your site to cross-site scripting (XSS) or other attacks. Helps maintain high-security standards by flagging outdated scripts.
   
   
8. SSL/TLS Version Check:
   Purpose: Ensures that SSL/TLS protocols are up-to-date and secure.
   Benefit: Reduces the risk of man-in-the-middle attacks by ensuring your site doesn’t support deprecated SSL/TLS versions. Essential for data integrity and user trust.
   
   
9. HTTP Methods Check:
   Purpose: Verifies which HTTP methods are enabled and if any insecure ones are accessible.
   Benefit: Minimizes risk by restricting access to dangerous HTTP methods (e.g., PUT, DELETE) that could enable attackers to upload or delete files maliciously.
   
   
10. Directory Listing Check:
    Purpose: Checks for any unprotected directories that expose file listings.
    Benefit: Prevents exposure of sensitive files and site structure, reducing the risk of targeted attacks. Securing directory listings is crucial for maintaining confidentiality and control over website content.
    
    
11. SQL Injection Test:
    Purpose: Tests for potential SQL injection vulnerabilities in user input handling.
    Benefit: Ensures database security by proactively identifying SQL injection risks, protecting sensitive information, and safeguarding user data.
    
    
12. Test Site Speed:
    Purpose: Analyzes the site’s loading speed and identifies potential bottlenecks.
    Benefit: Helps improve user experience and SEO by ensuring fast load times. A faster site enhances engagement and ranks better in search engine results, especially for mobile users.
    
    
13. Render-blocking Resource Test:
    Purpose: The Render-Blocking Resource Check finds CSS and JavaScript files that delay page rendering.
    Benefit: Helps you identify files to load asynchronously or defer for faster page speed. It provides optimization tips and can save a list of blocking resources for easy review in Site_Blocking_Resources.txt.
    
    
14. XSS (Cross-Site Scripting) Test:
    Purpose: Checks for vulnerabilities where attackers could inject malicious scripts.
    Benefit: Ensures safe interaction for users, especially in forms or areas where content may be dynamically displayed. XSS vulnerabilities can lead to data theft, unauthorized actions, and damaged user trust.

Additional Usability Features

• Suggestion Aggregation: Collects improvement suggestions at the end of the scan.
  
  • Benefit: Provides a consolidated list of actionable recommendations, making it easy to address all identified issues systematically.
  
  
  
• Optional Image Alt Tag Report: Allows the user to generate a report on missing alt tags.
  
  • Benefit: Simplifies accessibility improvements by identifying images needing alternative text, which benefits both SEO and visually impaired users.
  
  
  
• Automatic Cache Clearing: Clears the cache at the start, ensuring the most up-to-date site version is analyzed.
  
  • Benefit: Prevents outdated data from affecting test results, ensuring accuracy and reliability in reporting.
  
  

Overall Benefits
Using this tool provides a comprehensive and proactive approach to site maintenance. It identifies weaknesses and suggests best practices to improve site security, accessibility, and performance. By running this script regularly, you can:

• Stay Ahead of Security Threats: By identifying vulnerabilities before they’re exploited.
  
• Enhance User Experience: Through faster load times, accessible design, and stable functionality.
  
• Boost SEO: By maintaining a site structure, content optimization, and speed that search engines favor.
  
• Ensure Legal and Compliance Standards: Like GDPR, ADA, etc., by securing user data and meeting accessibility standards.
  

Overall, this script acts as an automated web audit, allowing you to maintain a high-quality site without manually checking each component. It saves time, offers insights based on actionable data, and helps keep your site in prime condition, ready to face both user expectations and potential security challenges.



Here are some key changes from the last version to this version. Last version, you had to open the file and edit in the URL of the site, then run the script. That is no longer the case, now to run it, open what ever terminal you are going to use, might it be PowerShell, Command Prompt, or using the terminal in VScode, what ever you use, Windows or Linux, it is as simple as just typing in this 1 command:



Of course, you don't need the

Please login to see this link Get registered or Log in

for it to work, but like I said in the notest from above, you must lead your URL with

Please login to see this link Get registered or Log in

or

Please login to see this link Get registered or Log in

I have updated the first post with the new file

If you find any bugs, please report it to me.

Can you put this into a batch file? I know nothing of python and wouldn't know where to start. I have hard enough time with html.

No, this script will not be ported to a .bat file. The best I can do is have it run on my server and put an input for you guys to enter the URL of the site you want to test. I more and likely won't be doing that as it is a pain to get it working, I know, I already tried and got it working this evening before deleting it. I removed it for I would need to rewrite the python script a bit, and after thinking about it, I don't want to open up a potential way for someone to exploit it and lag my server down since the script is quite extensive. Where it's at is probably where I will leave it as just an OS Python script anyone can download and use, but, if you like me to scan your site for you, I can do that and send you the output of the script. After you fix the areas that are mentioned, I can rescan for you. But that is up to you. I just buttoned up a lot of spots on my site, and I have sent Lonestar the changes I have done so he can put them into the next update.