Protocols for Server Downtime, Maintenance,
or 3rd Party Service Notification.

I) Monitoring and Notification

  1. Heartbeat monitors on all systems plus remote monitoring backups to auto-notify key partners when single units are down, total system down or failover engagement
  2. All parties who receive notification of downtime must check in with Network Admin or Developer Team Lead within 5 minutes of notification.
  3. All key parties are on call 24-7 365. Phones, pagers, and other alert devices must be on unless prohibited by law or regulation. Prior to exiting contact availability key parties must communicate with CEO or COO to identify the time period for offline status and ETA to return to communication availability.
  4. Key parties:
    1. CEO
    2. COO
    3. Lead Developer
    4. Network Admin
    5. Other Members of Developer Team
    6. SMO Team (for notification)
    7. VP Customer Service
  5. Errors or omissions
    1. In case of error that results in downtime all key parties must be notified regardless of time of day or location within 10 minutes of downtime error.
    2. All parties who receive notification must check in with Network Admin or Developer Team Lead within 5 minutes of notification.
    3. CS and SMO teams must communicate issues in forums, and via social media
    4. After Action report must be submitted to Community

II) Reboot Protocols

  1. Initiating Reboot (Unplanned)
    1. Initialing Developer must notify Network Admin, and lead developer.
    2. Trained Troubleshooting Developer or Admin must be physically at sever location to facilitate repair of any reboot issues
    3. All issues that occur must be logged in after action report for Developer repair
    4. CEO,COO, SMO Team and Customer service must be notified in email
    5. CEO, COO must be notified via Phone
  2. Initiating Reboot or Downtime (planned)
    1. CEO, COO, Network Admin and Lead Developer must be notified in writing for approval
    2. Once approved, notification to ArtFire community via staff announcements must be made in Forums.
    3. SMO, CS must monitor forums and Social media during planned downtime event
    4. After Action report must be submitted and published to Community

III) Service notifications from 3rd party providers

  1. All service notifications or potential events that are communicated by 3rd party providers are assumed to indicate potential downtime.
  2. All such communication must be forwarded in writing to all Key parties within 2 hours of receipt
  3. All such events will be assumed to create failure or error potential and will initiate failure preparation protocols
  4. In the event that any potential issue, maintenance , 3rd party provider service notification or utility, network wide event is anticipated the Development, CS, and Network Admin teams will initiate failure preparation protocols.
    1. Notification about potential event will be made to community.
    2. CS representatives and SMO will be placed on standby for addressing concerns or questions both internally and Internet-wide via scrapers
    3. A trained troubleshooting member of the development or network admin team will be placed on site with servers to address non-remote accessible issues
    4. CEO and COO will be notified.
    5. Upon termination of the issue window all parties will receive stand down notification from the team lead and community will be notified of the end of the potential error window.