Ecwid Status
[FIXED-PROBLEM] Facebook stores not displayed
We are sorry to inform you, at this moment we are having issues with display of Facebook storefronts. When you open your Ecwid shop at your Facebook page, you may see blank content.
We are already fixing this problem at the highest priority. We presume the problem is on Facebook side and are doing profound research to find a fix.
The issue is only with the display of Facebook storefront and does not anyhow impact your store data. It is all safe and secure, you can check your Ecwid control panel.
Your other storefronts, if you run any on websites or blogs, continue working as usual.
As a temporary solution, please try using Ecwid on your website, or if you don’t have a site - your simple store link (how to get it).
Will keep you posted on the progress here.
UPDATE, Feb 28, 9:45 AM UTC: We have localized the cause of the problem and are working on the fix.
UPDATE, Feb 28, 10:30 AM UTC:
Still working.
Further research has revealed that the problem is within Facebook, reported here:
https://developers.facebook.com/x/bugs/227236934134017/
Our team is working on the fix for Ecwid. We expect to apply it in around 30 minutes.
UPDATE, Feb 28, 10:50 AM UTC:
The problem has been fixed.
It is proved to be caused by Facebook failure: https://developers.facebook.com/x/bugs/227236934134017/
We have found a workaround for Ecwid. Now all Ecwid stores on Facebook are up and running.
We are sorry again for the troubles.
[PROBLEM-FIXED] Ecwid control panel access via social buttons
FEB 10
In the period from 15:16 to 19:42 UTC some of Ecwid accounts were having issues with accessing the control panel. It affected only social buttons login for a few of Ecwid users: If you clicked a button to sign in with PayPal, Google or Facebook, you might have received an error. The authentication through the regular login form (with email and password) were available.
We are really sorry for the troubles. The problem is currently fixed. It didn’t have any impact on Ecwid accounts, shops and data. Your storefronts continued working and receiving orders.
If you still need help with login to your Ecwid control panel, please feel free to ask for assistance at login-issues@ecwid.com
[SOLVED-PROBLEM] Ecwid Notifications Delayed
We are really sorry, at the moment Ecwid notifications to store administrators and customers are delayed. We are having issues with our mail servers which causes the notifications to be backlogged in queue. We are already fixing it and will keep everyone posted here.
Details:
Email notifications that Ecwid sends out to store administrators and customers may not be delivered starting from FEB 9, 22:32 UTC. All emails are there, just waiting in the queue. They will be dispatched as soon as we solve the problem.
The following notifications are queued:
- Order notifications
- Emails on registrations at Ecwid at https://my.ecwid.com
- Password recovery at https://my.ecwid.com
- Email verification from Ecwid control panel (if you have changed the email associated with your Ecwid account)
This issue does not impact orders and storefronts. Your Ecwid stores continue working and your customers can buy from you as usual. The new orders are recorded in Ecwid control panel, at Sales page, with all information. Inventory tracking is also working and updated automatically
We encourage all the store owners to check their Ecwid control panels for the new sales, placed between Feb 9, 22:32 AM UTC and Feb 10, 9:45 UTC, and contact the customers if necessary. If you need help with accessing your Ecwid account, please, feel free to contact us at login-issues@ecwid.com
Please, accept our deepest apologies for this trouble. We are urgently working on fixing the problem. We expect to fix it approximately in one hour. Follow up this post for the updates.
UPDATE, FEB 10, 6:00 AM UTC: we’re still working on the fix. The emails are intact and we’re working on relaunching their dispatch. We’ll be posting an update shortly.
UPDATE, FEB 10, 6:30 AM UTC: Still working. We had to launch a new server to handle the email. We’re testing the new setup to make sure that it’ll work perfectly.
UPDATE, FEB 10, 7:15 AM UTC: The testing is over, we’re going to start the emails dispatch in several minutes. The process may take some time. We’ll continue updating on the progress.
UPDATE, FEB 10, 8:00 AM UTC: We’ve started sending pending emails. For now ETA for delivery of all emails is 2-3 hours. Your patience is appreciated.
UPDATE, FEB 10, 9:45 AM UTC. FIXED: All emails have been sent. If you don’t receive some, please report to our support team.
We are sorry again for this situation. We will investigate how to avoid such issues in the future.
Thank you for staying with us.
NOTE: a small note to those who have SPF configured on their mail servers (what it is). You will need to do a small adjustment to the SPF record, please refer to this article for details: Adding SPF Record. Your hosting support should help you with this.
[SOLVED-PROBLEM] Ecwid Control Panel displayed in another language
JAN 30, 14:00 UTC
We are sorry to inform you, there is an ongoing issue with displaying Ecwid control panel in proper language for a part of Ecwid store owners. Some of you may see the control panel pages in a language that you haven’t used before.
We are fixing the problem in this very moment and will keep you informed. We do apologize for this.
If you sign in to your Ecwid account at https://my.ecwid.com/cp/CP.html and see another language, you can quickly switch it to any other language you need. Please refer to this article for instructions: http://kb.ecwid.com/w/page/29132769/Backend%20translations
The problem affects the control panel appearance only and does NOT anyhow influence your customers data, products and any other data, and your online stores. Ecwid storefronts are working as usual and should be shown in the languages you setup.
JAN 30, 16:00 UTC
UPDATE: The issue has been fixed. We’re sorry again for the troubles.
Public report on Ecwid unavailability on December 16th
On December 16th Ecwid service was not available for up to 33 minutes for about 50% of the stores. The issue occurred during the maintenance procedure that was supposed to be seamless. During the outage both storefronts and control panels were not available for about 50% of Ecwid accounts. Except for service unavailability there was no data loss.
Please, accept our deepest apologies for this issue. Below you can find the detailed explanation of what had happened.
We understand that our customers rely on us to keep their stores continuously available, stable and to provide smooth and fluid user experience. For the last months our operations team has been working on a huge task. We migrate Ecwid into private networks inside Amazon Web Services (AWS) called Virtual Private Cloud (VPC), as keeping backend servers in a private network without direct access from Internet is the most secure way to protect the customer’s data. And eventually we were able to make this improvement for all of our customers.
Ecwid cluster is a complex system which consists of many interconnected subsystems. The migration to VPC consisted of many steps. On each step we carefully moved one subsystem, tested it and switched the other parts of cluster to it.
One of the cluster subsystems is the store routing system. It is responsible for routing the requests from frontend servers to the proper backend servers inside Ecwid cluster. In order to migrate the system seamlessly we completely replicated routing system state inside protected network and switched the cluster to the new routing system. As planned, the migration was performed without service interruption.
Here in Ecwid we use configuration management solution called Chef for configure our servers. Among other things it allows us to make sure that all our servers are configured in the same way. Chef solution deserves a separate blog post, here we can say that Chef to a great extent enabled us to migrate service into VPC without service interruptions.
On one step of moving into VPC we configured the frontend servers with Chef. However during the migration of routing system the local configuration was updated, at the same time the configuration in Chef was not updated. A bit later Chef reverted the routing configuration back to the state it is aware of. The update stayed unnoticed by operations team.
Another problem was that we didn’t stop the old routing system. All that time we actively migrated Ecwid stores between servers inside Ecwid cluster. A week later the routing records inside old system became incorrect for about 50% of stores. As stated above the frontend server’s configuration pointed to this old routing system and we were not aware of that.
A week later, on December 16th, we performed update of frontend application credentials. It included reloading of frontend servers. Usually we can reload frontend applications without service interruption. In order to keep service stable we reload them one by one. This time initially the configuration of only one frontend node was updated and restarted. The log records were checked and no abnormal records were noticed. Then the other frontends were reloaded as well.
Since frontend applications were configured to old routing service, they loaded outdated routing state for stores moved since routing system migration. The frontend servers routed requests for such stores to the wrong backend nodes. Incorrect nodes failed to respond properly. In a minute our monitoring system triggered the alert and engineers started to investigate the outage. It took 33 minutes to find out the origin and fix the issue completely. We are very sorry the issue mitigation took so long.
After the issue was mitigated we run a meeting. We discussed the issue in details and identified the steps we need to perform in order to prevent such issue in the future.
First of all we surely updated the frontend configuration in Chef and made sure that it distributes properly.
Secondly, we stopped the old routing system. If for some reason we are going to get misconfigured fronted application, it won’t run and let us know that the routing table is unavailable.
Thirdly, we updated the upgrade procedure using our new knowledge. We believe the new procedure is much safer, than the one we followed previously.
Besides that we planned the improvement in our changes monitoring system. So next time we will be aware that the service critical files are updated either manually or by any automation tool.
We believe these measures protect Ecwid from the whole class of potential issues. Please accept our apologies once again. Please be sure we are doing our best to keep Ecwid the best solution for you.
[FIXED-Outage] Some of Ecwid storefronts and control panels are not loading
Jan 22, 04:00 PM UTC
We are very sorry, one of Ecwid servers is having troubles right now: a part of Ecwid storefronts and control panels are not loading.
Our engineers are fixing the problem. We will be publishing updates here.
Please accept our apologies for this downtime. All your data will be kept intact and safe.
UPDATE, 4:50 PM UTC
We have found a fix and are implementing it right now. We’ll let you know about the results shortly.
UPDATE, 5:20 PM UTC
The services have been restored. The storefronts are UP, you should be able to access your control panel. We continue monitoring the system to make sure everything is working.
[COMPLETED-MAINTENANCE]: Ecwid update on Jan 17
On Jan 17, at 12:00 PM (UTC), we will be releasing an Ecwid update.
During the work, the control panel will be on hold for several minutes, and display a warning about Ecwid update. Therefore if you plan to do any modifications in the control panel, we recommend completing and saving all the changes prior to the specified time (you will see a warning about upcoming update several minutes before it starts).
The update will not anyhow touch Ecwid storefronts. Your shops will continue functioning as usual, and your customers should not notice anything. Your settings and data will be kept intact as well.
[FIXED - Outage] Ecwid storefronts and control panel are not loading
Jan 9, UTC 08:00 AM
We are very sorry, Ecwid is having troubles right now: all of the storefronts and control panels are not loading.
Our engineers are already fixing the services. We will be publishing updates here.
Please accept our apologies for this downtime. All your data will be kept intact and safe.
UPDATE Jan 9, UTC 08:20 AM:
The services are being restored. Most of Ecwid storefronts should be displayed by now.
UPDATE Jan 9, UTC 08:50 AM:
We’re still working on fixing the problem. We’ll come up with an update shortly.
UPDATE Jan 9, UTC 09:20 AM:
Unfortunately there’re still problems with accessing the storefronts and control panel. We’re doing the utmost to fix this soon. We’ll keep you updated.
UPDATE Jan 9, UTC 10:10 AM:
Ecwid is up! The problem has been fixed. All services are restored. Ecwid storefronts should be up and running, Ecwid control panel is now accessible.
All data (products, orders, your store settings, etc) are intact. Your Ecwid stores continue functioning as usual.
Thank you for your patience. Again, we deeply apologize for this situation. We will be conducting a research to understand why the issue took place and how we can avoid it in the future.
[SOLVED] Issue with displaying our Knowledge Base
Jan 8 2014, 11:00 AM, UTC
We are sorry, our Knowledge Base at http://kb.ecwid.com/ is down at the moment. The third-party tool, that we’re using on this resource, is having problems and not able to deliver the content.
We are already working on the issue. It is expected to be fixed shortly (within one-two hours).
If you try to open http://kb.ecwid.com/ and see an error, please check back later or contact our support team on the forums. We will give you the information you need.
Please, don’t worry, the issue has no bearings on Ecwid services. Your stores continue functioning as usual.
UPDATE: Jan 8 2014, 13:00 UTC
The knowledge-base site http://kb.ecwid.com/ was restored and is now up and running. Feel free to use it to search for many helpful tips and articles.
[Problem - Resolved] Settings of payment methods were reset in some of Ecwid accounts. We’re restoring the settings.
Dec 20 2013, 11:08 UTC . After a recent update on a few of Ecwid servers on Dec 19 10:00 UTC payment method settings were reset in a part of Ecwid stores (less than 1% of Ecwid stores are affected). As a result, online payment methods (credit card and other real-time payments) were disabled and are not displayed at checkout in the affected stores right now.
The issue affected the ‘Payment’ section of store settings only, all the other settings and data are kept intact.
Please check your payment settings in your control panel at http://my.ecwid.com (System settings → Payment). If they have been reset, you will see the warning icon in the method list: 
. In this case, please re-enter your credentials.
Our engineers are restoring the settings centrally from backups, so all your payment settings will be returned automatically and payment methods will work OK. We expect the problem to be completely resolved in an hour.
Please accept our apologies for this issue. We will inform you of the progress.
UPDATE: we restored payment method settings in all affected stores. Everything should work fine.
We’re very sorry for the troubles caused by this issue. Our engineers will continue investigating this issue to prevent it in the future.
