The Fiasco Called Marketplace

Things were looking pretty promising there for a bit .. a very short bit. Direct Delivery was released, and in spite of a few disappointing first issue bugs, things seemed to be smoothing out. But then the hammer dropped on Maxwell’s head … and since then the Lab and the Commerce Dev Team have been making some very strange decisions. Strange as in “no responsible company would do that” kind of decisions. Decisions that not only throw their customers right under the bus, but seem to say they accept no responsibility for any errors in the Marketplace because they are “Legacy Issues”. That’s all well and good that the errors were happening before Direct Delivery was released, but their reappearance and massive uptick in occurrence AND the fact that it IS the Commerce Dev Team’s responsibility to ensure the Marketplace works only prove that the Commerce Dev Team needs to get hold of this now and stop trying to pass the buck onto the Merchants. Besides, the Merchants can’t fix it anyway. Allow me to explain …

Zee Bork! Zee Bork!

In order to understand exactly what is “Borked” (or broken) on the Marketplace, I’m going to have to dig into some fairly techie type stuff. But before I do that, you’ll want to get familiar with the JIRA Issues that have been created about it. There are currently two, although for some strange reason they have been marked as duplicates when in fact they are not the same problem. At any rate, the first and most heavily populated Issue is WEB-4587 : Incorrect items showing up on my marketplace store from other merchants but with my listing texture and vice versa. The second issue created just yesterday is WEB-4601 : Related items from another merchant’s store are appearing on my listings.

In layman’s terms this means that when you (as a Merchant selling on Marketplace) look at one of your Product Listings, you see things that are wrong with it. The “things wrong” can be wrong images, wrong price, wrong description, wrong maturity ratings, wrong Creator Name … all sorts of wrong stuff. What makes this so annoying is, the wrong stuff isn’t stuff you put there. Oh no, it’s stuff the Marketplace decided to put there. However for some reason the Commerce Dev Team seems to think you (as a Merchant with literally loads of time on your hands and nothing better to do) will leap at the chance to edit all those listings and fix them yourself. (What HAVE they been smoking?)

Even deeper into this issue though is the fact that you can’t edit them all. Some of the listings that appear in your store are not actually owned by you. They may have your images attached, or they may have some of your related items or description, but they are actually “owned” by someone else. If you don’t own the listing then you can’t delete it or edit it. So you can’t remove it from your store, you can’t fix it and you can’t contact the Merchant that owns it because they’ve left Marketplace a long time ago.

Now, lets dig into the deeper technical side of things. (And for those of you without electrons orbiting your heads, I will attempt to make this easy enough to understand so your eyeballs won’t implode.)

How A Product Listing Fits Together

There is a big scary computer word attached to this problem; that word is “Database”. If you’re not really familiar with what that is, think of it like a filing cabinet. A Database is nothing more than a collection of data, like the pages and folders in a filing cabinet. In the case of the Marketplace, the data stored in the filing cabinet is everything about the Products, the Merchants and any other miscellaneous stuff that it needs to know about in order to operate. However, unlike a filing cabinet in which all the pages for a particular Client are all kept in the same folder, the Marketplace stores the Types of data in separate folders. For example there is a folder for all the product images, there is a folder for all the descriptions, there is a folder for all the related items … so on and so forth. Exactly what types of data are kept and what folders there are, I don’t know exactly. However that’s not important to understand the problem. All you need to know is that the parts (or “Components”) of a single Product Listing are spread across a number of different folders. (Computer geeks refer to these “folders” as “Data Tables”.)

One special type of folder in the filing cabinet is one we’ll call the Master Product Folder. Each Product listed on the Marketplace has a single sheet of paper that lists all the Components for that Listing. It contains the page numbers (called the Keys) of the Images for that Listing. It contains the page number of the Description for that Listing. It contains all the page numbers (all the Keys) needed to pull the proper pages out of the other folders and build a complete Listing.  So in order to display a Product Listing, all the Marketplace has to do is grab the page from the Master Product Folder for that product, then open each of the other Component Folders, fetching the proper pages from each of them as detailed on the Master Product Page. (The Master Product Page is kind of like the assembly instructions in that it gives you pointers to where you can find the parts needed.)

Part of the problem here is that sometime when the Master Product Page was being saved or retrieved, or perhaps some other time when a customer was just looking at your product, the page numbers written on the Master Product Page got corrupted. (“Corrupted” is a computer word for all f**ked up.) So when the Marketplace starts piecing together the Components of the Product Listing to display it, it pulls out some wrong pages from some of the Component Folders. The result is you will see the wrong images, or the wrong price, the wrong description … you’ll see something wrong in your Product Listing. In computer lingo this is called a “Forward Linking Error” and it means when you follow the links (the page numbers) going forward from the Master Product Page to the Components, some of those links are wrong and you wind up retrieving the wrong Components. This is the first type of error, but not the only one.

We Got You Coming And Going

If there’s a Forward Linking Error, you’re probably thinking “what about Backward Linking Errors?” And you’d be right to think about that because the Marketplace has those too. On every page in every folder there is information written that not only tells the Marketplace which Merchant owns it, but also which Product it belongs to. For some reason, when either you were saving a Product Listing, or maybe when someone was just looking at your Product Listing, or just because the evil digital fairies decided they hate you this month, the “backward links” … the Owner and Product information written on a page got corrupted. So if you start with, for example, all the Product Images that you own then follow the backward links to find all the Products you own from those, you will not only get your Products but you will also get some Products owned by other Merchants. As you can see, if you’re trying to find all the Products from a Components Folder, you will wind up with some bogus Products because some of the backwards links are corrupted too. In short, you’re screwed going either way, forward or backward.

But I’m Only Looking At My Products!

You would think that these backward link errors wouldn’t bite you since you’re only displaying the Products in your store, but there’s a little “Gotcha” that a lot of database newbies forget about … and it bites them every time. It has to do with a database operation called a “JOIN”. (You may have run across this term a few places and didn’t know what it meant. Well get prepared to get educated .. or keel over from confusion .. one or the other.)

Remember that it’s a computer assembling all the Components to make your Product Listings. It does this by going through the Master Product Folder reading all the pages that belong to you, then retrieving (called “Joining” in computer terms) the pages from the Component folders that are listed on the Master Product Page. When it gets all done, it should just have the Components for the Products that belong to you, right? Well .. almost. And here’s the tricky part.

The reason the computer can “JOIN” the Component Pages together is because the same page number and Folder Name are written in both the Master Product Page and the Component Page. The computer looks to see that those two match and adds the page to the Product it is assembling. But here’s where the “Gotcha” gets ya. If, for example, one of the images it pulls back is linked to someone else’s Product (due to a broken Forward or Backward Link) then the computer will also add THAT Product to the list of Products it needs to retrieve. (This can happen all too easily if you are using the wrong type of “JOIN” when telling the computer how to assemble the Product Components.) All of a sudden, someone else’s product is now being included in your store because they have a product that is accidentally linked to one of your images.

But It Doesn’t Show Up Under Manage Listings!

That’s correct, and it won’t either. When the Marketplace is building the list of products for display in your Store, it pulls up all the Master Product Pages AND all the Component Pages that are owned by you. Thus a broken forward or backward link can wind up causing the inclusion of Products that don’t belong to you even though they may accidentally contain one of your images or some other Component. When the Marketplace is building a list of the Product Listings for you to Manage, it only looks at Master Product Pages that you own.

Caveats and Emptors Too

What I’ve described above is how I understand the problem based on what I’ve been able to glean from talking to other Merchants, studying broken pages and from reading the words that have come across my desk from CommerceTeam Linden. (And I’m still glad they are communicating more this time. I’m just disappointed that they’re not taking full control of this Cross-Linked Listings fiasco.) What I do not have is a full Database Schema of the Marketplace (although I wish I did) and I do not have a full set of SQL Queries as implemented in the Marketplace (although again I wish I did).

This Is An Old Bug With A New Lease On Life

This error has indeed existed since a long time ago but has been steadily growing worse as time has gone by. It’s my belief that due to the overwhelming load put on the Marketplace by the recent rash of Product Edits (which is where the broken forward and backward links most likely are occurring), this error has become a LOT more prevalent. Also because we are seeing the errors in both Direct Delivery and Magic Box listings, no doubt the error is some sort of deeply rooted Multi-User Error … the type that only occurs in heavily loaded systems where multiple users are accessing it at the same time. These are NOT fun errors to find. It could even be due to errors in the particular version of SQL they are using or in the particular version of RoR, or any of a number of other faulty systems. In all honesty, the cause of this error might not ever be found, not unless they get a serious DBMS Guru involved. (And Gurus ain’t cheap.)

But Wait, There’s More!

The one thing that sticks in my craw the most about this situation is the Commerce Dev Team’s belief that the Merchants ought to be fixing up these errors. I’m sorry but that is not our responsibility. I will grant you that finding and fixing the real cause of the error is probably going to be tough. But this was not an error caused by the Merchants, and as has been shown by several others, it’s not even possible for the Merchants to fix all the errors. However it IS possible for Linden Lab and the Commerce Dev Team to fix them, or at least fix most of them.

Remember that each Component contains information about who “Owns” the “page” … the data for that Component. When assembling a Product Listing, if the Ownership of any Component conflicts with the Ownership of the Master Product Page then you know that the link to the Component is broken. So it should be a fairly simple matter for LL to run a “Consistency Check” to walk through all the Components and single out those that have conflicts in Ownership. Depending on the type of Component, the action they take from there will be everything from simply deleting the link between the Component and the Master Product Page all the way up to finding and replacing it with the correct component. (There’s a good chance that each Component also contains information about when it was last updated. If this is true then it should be fairly simple to find the parts that are unlinked and have the same or similar update times and dates and hook them together. At the very least you’ll get the right Ownership fixed up even if you do wind up sticking the wrong picture with the listing.

Final (or is that Fatal) Thoughts

I do not like to be a Doomsayer … or a “Sky is Fallinger” (whatever!) but I seriously do not see good things coming out of this unless Linden Lab ponies up and does this fix on their own. Sure, we’ll be happy to help in reasonable ways, but we DAMN sure won’t be doing slave labor, spending hours and hours fixing database problems caused because they chose the wrong software as the basis for Marketplace, or the wrong version of SQL or whatever it turns out to be. They get paid a commission to provide a service. Just like the phone company and the power company and the grocery store down the street. But if one of those other service providers makes a mistake and screws things up for their customers, they are expected (both philosophically and legally) to rectify that error. When it comes to a situation where they are costing their customers money because of their mistake, that’s an extra incentive for them to fix it. Legally we are not entitled to sue them for lost income or “damages”, that’s well spelled out in their varius legal agreements and contracts. But legal and “damn well ought to” are two different things, and in this case … they damn well ought to.

We suffered enough having to redo all of our listings when they prematurely shut down XStreet SL and turned on Marketplace. We’ve done a lot of slave labor related to Direct Delivery too. We are not going to continue doing slave labor to fix this error .. not this time. I have the distinct impression from my talks with other Merchants, both those still in Second Life and those that have abandoned ship for other Grids and VWs, that unless this can be made to go away with a minimum of fuss, a lot of people are going to jump ship again and leave SL for good. Perhaps enough that SL will drop below that critical mass of talented creators and inventors and paying customers to make it profitable for Linden Lab.

I’m not threatening either. I’m just being honest about my sense of what the mood is like “out here”. Maybe I’m asking the wrong people and there is a flock of folks that don’t give a flip about how this plays out … but I don’t think so. I guess though, only time will tell.

Visit the DGP4SL Store on SL Marketplace


10 Comments on The Fiasco Called Marketplace

  1. Argus Collingwood on Sat, 31st Mar 2012 3:09 PM
  2. Darius, yep, we have a fubar database and yep, we can’t fix it. If they did a re-index and it is still fubar then the database mavens need to apply their magic and clean it then find the root cause. We watch and wait while sales decrease. I have converted a few items to DD and so my store is mixed so far so good but I will not be dumping more data into a system that is unstable. I will wait. Great read BTW, hope Rodvik sees it;-)

  3. Czari on Sat, 31st Mar 2012 3:17 PM
  4. Excellent post! I’m one of the merchants with a borked link to someone who is no longer a Marketplace merchant and most likely no longer in SL unless they have a hidden profile. What irks me the most, in a long list of current irkdom, is that even if I pulled all my listings down until this is fixed, the borked links to vanished people’s products/site would still be there keeping my MP store “open.”

    We are decidedly not amused.

  5. David Kaye on Sat, 31st Mar 2012 5:49 PM
  6. Linden Labs? Second Life? Gosh, I hadn’t thought of that company in years. Are they still around? Nobody talks about them anymore. Last time I was on there there were gatherings of avatars talking about Alcoholics Anonymous and Jesus. Wasn’t my thing so I put my avatar away and left. Thanks for the nostalgia, mates.

  7. Sapphy Rotaru on Sat, 31st Mar 2012 8:41 PM
  8. We can all pray that now you have worked out the problem for them &, I think, very accurately summed up the mood, LL will use your wise words to dig themselves out of this very deep hole they are in.

    I have over a thousand listings on the marketplace &, as yet, I can’t find any problems with any of them BUT I would be very relieved if they closed marketplace sales until this & the other showstoppers are completely resolved. If there was a way to leave it open so that listings could be viewed like a catalogue & store links followed to in world stores then even better. If left as it stands it can only damage merchants credibility & reputations even further and it does damage us all, not just those with the borked listings.

    We suffered enough from the delivery problems, the majority of non merchant customer’s don’t understand that we are at the mercy of LL, they just think merchants are incompetent or trying to rip them off. I am sure we have all had those reviews saying DONT BUY THIS, I ORDERED IT & IT DIDNT ARRIVE, or the note card saying SEND ME MY MONEY BACK! Only a very small percentage of SL residents read or even know about the forums, there will be many merchants who don’t realise they have a problem & even more customers who don’t know what is really going on.

    I have been angry before with problems on the marketplace but this tops them all & it makes me very sad to see the world I love being damaged by such inconsiderate & unbelievable behaviour from the marketplace team. Their scant replies have only so far proved that they are not listening to all the facts & are certainly not grasping the gravity of the situation. I will continue to publicise the Jiras & blogs on the situation in the hope that customers will understand that the merchants are doing their best because even with no affected listings i realise that I am part of a community & that any negative impact affects us all.

    Thank you Darrius for explaining things in a way that, even I, as a complete techno-incompetent can understand. A well written & very informative post.

    […] Darrius Gothly: The fiasco called Marketplace Rate this: Share this:TwitterDiggRedditFacebookMoreEmailPrintLike this:LikeBe the first to like this post. This entry was posted in Opinion, Second Life and tagged Marketplace, SLMP by Inara Pey. Bookmark the permalink. […]

    […] so far, ok I may be underplaying the issues there. Darrius Gothly has a blog post entitled: The Fiasco Called Marketplace which has an in depth look at the issues. I’m not going to take an in depth look at the […]

    […] Darrius Gothly: The fiasco called Marketplace Rate this: Share this:TwitterDiggRedditFacebookMoreEmailPrintLike this:LikeBe the first to like this post. This entry was posted in News, Opinion and tagged Direct Delivery, SL Marketplace by Inara Pey. Bookmark the permalink. […]

    […] Darrius Gothly: The fiasco called Marketplace Rate this: Share this:TwitterDiggRedditFacebookMoreEmailPrintLike this:LikeBe the first to like this post. This entry was posted in News and tagged Direct Delivery, SL Marketplace by Inara Pey. Bookmark the permalink. […]

    […] Darrius Gothly: The fiasco called Marketplace Rate this: Share this:TwitterDiggRedditFacebookMoreEmailPrintLike this:LikeBe the first to like this post. This entry was posted in Opinion, Second Life and tagged Direct Delivery, SL Marketplace by Inara Pey. Bookmark the permalink. […]

    […] Darrius Gothly: The fiasco called Marketplace Rate this: Share this:TwitterDiggRedditFacebookMoreEmailPrintLike this:LikeBe the first to like this post. This entry was posted in Opinion and tagged Direct Delivery, SL Marketplace by Inara Pey. Bookmark the permalink. […]