Tuesday, December 11, 2012

Four meditations on bad design decisions

I've recently been doing a major rewrite of one of our core engine systems, the graph that we use for our visual scripting language Flow. Taking it from something that looks like this:

To something that looks like this:

A major rewrite like this is always a humbling experience. When you have to rewrite your own code, every bad decision you made comes back to haunt you. And you don't have anybody else to blame them on.

As if facing your own inadequacy wasn't enough -- rewriting an existing system is always harder than writing one from scratch. When you write a new system you start with a blank slate and can do whatever you want. When you rewrite, you are constrained by what the old system did -- at least if you want to maintain any kind of backwards compatibility.

In addition, a new system can be written iteratively. You can start with a very small, simple system, release early and get feedback. Based on that feedback you can tweak the system. You don't have to think about adding features until you have a good stable base.

When you are doing a rewrite you can't release the new system until it is at least as good as the old one. Otherwise, your users will question why you have spent all that time working on a system that is worse than what you had before. And they will be right.

So a rewrite forces you away from the comfortable land of early releases and quick iterations and into the ugly old waterfall model.

With the power of hindsight, I'd like to reflect a bit on four design mistakes I made when I wrote the first version of the system that made this rewrite a lot harder than it could have been.

Don't use strings for non-text things

Strings have one really good use -- to hold pieces of text that either gets displayed to or inputted by the user. All other use of strings should be regarded as suspicious.

Strings are scary because they are both ambiguous and powerful. Does "a/b.txt" and "A//b.txt" represent the same path? Hard to tell. But maybe you can use case conversion, search and replace and some regular expression monstrosity to figure that out.

If you are doing that kind of string manipulation in any part of the code that is not directly related to user input or output, it is a clear warning sign that your code might be too "stringified".

The most obvious example stringified code is the use of "stringly typed" data, for example, storing a date as the string "2012-12-09". But the problem with strings can also manifest more subtle ways.

The original version of Flow used strings to identify connectors, both internally (as a representation of the connection) and visually (to show the name of the connector):

As a consequence, a Flow node couldn't have two connectors with the same name, and a connector couldn't be renamed (even visually) without breaking all existing connections.

In retrospect, rather than having a single Name property, it would be much better to have separate Id and DisplayName properties. The Id would be a GUID that uniquely identified the property, and the DisplayName would be a (localizable) name, suitable for displaying to the end user.

Using names/strings as identifiers has bitten me in other ways as well. In one system I knew that the names had to be unique (because that is how the script would refer to the objects) so I thought it would be safe to use them as identifiers. What I didn't consider was that there could be situations when there temporarily were two objects that had the same name. For example, if the user had created a rock object, and wanted to create a rock_small object -- as she was half-way through typing that name, there would suddenly be two objects named rock. This created problems for the system.

Lesson learned, I now avoid using strings as identifiers.

When in doubt, you should opt-out

Every system acquires features over time. That is good of course. Those features make the system more powerful and easier to work with.

But among the good features there are usually a few that don't feel quite right. That don't really fit into the design of the system. You can do them of course. You can do anything.

But usually it is best not to. Most of the time when I have added a feature that didn't quite feel right, I have regretted it later. In retrospect it would have been better to try to find a different way of doing what the users wanted that was more natural to the ideas behind the system.

An example: Users of Flow wanted some way to specify the order in which events were triggered, when multiple connections are connected to the same Out connector. This is needed in some situations, for example you may want to make sure that a unit is spawned before it is used.

In the old version of Flow, this was implemented with a context menu on the connection where you could select if it should be a "Do First", "Do Last" or "Do Normal" connection.

This solution never felt 100 % right to me. It was hard to find a good intuitive way to visually represent the "Do First" and "Do Last" connections, and as a result the Flow graphs became harder to understand.

In retrospect, it would have been much better to avoid this feature and wait until I had come up with the more elegant alternative: a sequence node that triggers each of its outputs sequentially:

Be explicit or you'll miss it

Writing code where a lot of things happen implicitly feels great -- to begin with. It is amazing how much you are able to do with just a few lines of code.

But in my experience, implicit code almost always ends up more costly in the long run. It is harder to understand, harder to debug and harder to change. It tends to lock you down in a "local minimum" that can be tricky to come out of.

In Flow, a lot of things are done implicitly. The definition of a Flow node is just a simple C# class:

[Category("Animation")]
public class AnimationEvent : Node
{
    public InVariableUnit Unit;
    public StringVariable Event;
    public InEvent In;
    public OutEvent Out;
}

Through reflection, Flow finds out the members in the class and their types and automatically generates Flow nodes for them. This process involves some ugly string processing (bad decision #1), such as stripping In and Variable from the type name to find the underlying type of members. Reflection is also used to serialize the graphs.

While it is nice to be able to express a node so concisely, there are also a lot of problematic consequences. For example, since the class names get serialized, we can't change the names of classes or properties without breaking the ability to load old files. Also, we have to use some really ugly C# hacks to make sure that the reflection system always returns the members of a class in the order they are declared in the file (so that we can control the order of the connectors).

In retrospect, it would been much better to avoid all this clever reflection stuff and instead just define the node types in configuration files.

Avoid the road of complex code

There is some code that needs to be complex, because it is dealing with fundamentally tricky stuff (like computational geometry) or because it needs to run really, really fast. But in all other cases, complexity is just a cost.

If your code starts to feel complex and hard to keep track of, it is a sign that you are probably doing something wrong. And if you are not careful, you may lock yourself in, so that when you write the next version of the system, you have to recreate all that complex behavior in your new, simpler system. You have to deliberately make your code uglier.

The old version of Flow had a way of "folding" nodes. You could select a number of nodes, group them, and then "fold" the group, collapse it to a single node.

The system had a lot of really hairy code for dealing with this. The code takes a bunch of nodes and creates a new node from them, with connectors matching only the external connectors of the collapsed nodes. it also keeps track of the internal nodes and their connections so they can be recreated if the node is later "expanded".

As you might imagine, this was complicated further by the need for connector names to be unique (see bad decision #1), which meant that some of the external connectors in the new node had to be renamed (since they could come from different internal nodes that had connectors with the same name). So a mapping table was needed to keep track of these renames. Obviously a bad idea, but once you have started down the path of wrongness, it can be hard to turn around.

The new version handles this a lot better. Collapse and expansion is just a visual feature. There are no new nodes created and no other strange things happening to the data, the visualizer just chooses to draw the data in a different way when it is collapsed. In retrospect, that is a much better choice.

That is all, four simple lessons
to guide your future coding sessions
now let your code be light and merry
until its time for Charon's ferry

91 comments:

  1. I don't think it was a bad idea to use reflection, as a user I'm not fond of using configuration files when everything I need was in C#. Couldn't you achieve the same effect with some custom attributes on fields?

    Also, once I had to use the fields in the order they were defined and I believe if you sort by the MetadataToken you could achieve that.

    ReplyDelete
  2. You can, and maybe it is "the C# way"... but I'm not very fond of that approach. I think it muddles responsibilities.

    The end result if you go down this road is that simple classes, such as Vector3 get a lot of attributes related to serialization, how the class will be displayed by the GUI etc. It doesn't feel like they belong there. What if some project that doesn't even have a GUI wants to make use of the Vector3 class? What if we have different GUIs that want to display the Vector3 class in different ways?

    ReplyDelete
  3. Maybe you can separate these situation specific attributes as another class, example of a "view" class for Vector3:

    public class View
    {
    public object Target { get; set; }
    public virtual Draw();
    }

    [CustomView(typeof(Vector3))]
    public class Vector3View : View
    {
    public override Draw() {
    Vector3 v = (Vector3)Target;
    Display(Vector3.x);
    Display(Vector3.y);
    Display(Vector3.z);
    }
    }

    You can scan the assemblies and look for these CustomView atributes, effectively decoupling rendering information from the Vector3 class itself.

    This is a common pattern used in Unity 3D, take a look at http://blogs.unity3d.com/2012/09/07/property-drawers-in-unity-4/

    Does that makes sense?

    ReplyDelete
  4. - Don't use strings for non-text things
    Having a DisplayName separate from the Id field makes total sense. Making the Id field a Guid is only one solution though. In many cases it's still okay and much more readable to use a string for the Id field as long as it doesn't change.

    - When in doubt, you should opt-out
    Sometimes features are not done right the first time. But I think a rewrite is the perfect time to re-do those features properly. Opting out probably wasn't the right thing to do a the time because it's a feature users wanted. Doing it better may have been worth the time.

    - Be explicit or you'll miss it
    I agree that being explicit is often better. There's a use for every tool though, and sometimes implicit is the right tool. Since it resulted in hacky code, in there probably is a better design.

    - Avoid the road of complex code
    I remember reading a quote once that said something like:
    "There are no complex problems, only complex solutions"

    ReplyDelete
    Replies
    1. For Ids there are two cases I think. One is when you have to uniquely identify user created objects. In that case I think that GUIDs are the easiest way and also the way that most clearly states the intended purpose.

      The other case is when the Id's essentially acts as enums -- specifies one of a limited set of predefined options. In that case I agree that it is better to use a predefined static string (e.g. "bold") to identify the option when you save the file to disk (otherwise your save files will be as incomprehensible as RegEdit).

      But I still don't think it is a very good idea to use the string "bold" internally to represent bold text. It is neither clear or readable and likely to cause a lot of confusion. I would much rather read the string "bold" from the file, convert it to an enum (Styles.Bold) and use that in the code.

      Delete
  5. Nice post, Niklas! It would be really interesting to know more about real world uses of Flow.

    What seems inconvenient to me in Flow's approach to implementing game logic is it's event driven basis. As I understand Flow's program might have several entry points - events which connects the program to the engine (or lua scripting layer). Also user is able to make really complex networks as some nodes may have a lot of connections and making cycles in graph is also acceptable. It seems that it isn't so easy to debug such a tangle of nodes. User must be aware of event driven programming pitfalls to make relatively complex programs. But the language is targeted to non-programmers.

    That is why I'm so interested in real use cases of Flow. What is it used for in real projects (AI, game specific logic, camera's behavior and so on)? How do you (or engine users) manage Flow's programs in large projects? Is it useful for non-programmers?

    ReplyDelete
    Replies
    1. Flow is heavily used by artists, animators, level-designers and other non-programmers. It is not intended for use by programmers (they work in Lua).

      I'm not sure why you think the event model is complicated or what you would like to see instead. Note that Flow nodes are not "active". They don't do any processing unless triggered by an event. So the flow is:

      input event (physics collision, animation, etc) ---> some logic --> result (play animation, effect, sound, etc)

      That is pretty straightforward.

      Flow is not intended for "programming in the large" or for solving really tricky problems (for that you want real programmers and a real programming language). It is for connecting things up, with some simple logic.

      Delete
    2. I thought it's intended to be used in more complex cases which programmers solve with FSMs or Behavior Trees usually. But they are more suited for active objects. It definitely makes sense to use this approach in such simple cases like triggers, I agree with you.

      Thank you for the explanation

      Delete
  6. This is somewhat off-topic, but I'm curious how you evaluate the nodes with respect to multiple output connections. If a content creator doesn't use a sequence node to specify the order, does that imply they don't care or don't know which outputs will be executed in which order? Is there a reason to allow this ambiguity i.e. should you enforce that every output can only have 1 output link and if they want more, they should output to a sequence node?

    Additionally, do you evaluate nodes in a depth or breadth first way? That is, if you have a sequence node with 3 outputs, do you evaluate the nodes connected to output 1 then 2 then 3? Or do you evaluate the node connected to output 1 then evaluate its outputs, etc.?

    Some actions may also have logic that occurs over time and cannot finish when initially evaluated. Have you run into content creators finding themselves in situations where race conditions occur with their logic? Do you have best practices or standards for dealing with this?

    Sorry for the amount of questions all in one post. I'm interested if you encounter these types of problems and how you handle them. Thanks!

    ReplyDelete
    Replies
    1. If you have multiple nodes connected to the same output event they are triggered in unspecified/random order. This works well as a default, because usually the order doesn't matter. If you trigger a sound and a particle effect as the response to some action, it doesn't matter which happens first, since they both happen in the same frame. In the few cases where order DOES matter, you can use a sequence node.

      I use depth first evaluation.

      Nodes in a flow network are not "active". I. e., they have no update() action. They only react to impulses. But they can be connected to "active" external system. For example, the Delay node queues an event with a time system and gets an impulse when the specified time has expired.

      Since everything happens with "impulses", it is pretty easy to reason about, which reduces the risk of confusing race situations.

      Delete
  7. thanks for sharing this guided information to us. it was really helpful.

    also introduce norton antivirus with the norton product key to protects your data with some advanced features from malware attacks and viruses.

    https://i-norton.com
    Norton setup
    norton.com/setup
    www.norton.com/setup
    help.norton.com 
    enter Norton setup product key
    norton removal tool
    Norton product key
    enter norton product key code to activate
    Norton setup with key
    norton setup enter product key
    norton setup enter key code 25 digit

    ReplyDelete

  8. To install office setup you have to select the downloaded file otherwise insert the office setup CD disc. If you use the CD disc then you have to enter the Office Product Key for authorizing it. After selecting the downloaded file you have to run or setup this file on your computer.


    office.com/setup
    office.com/setup


    Install Norton Antivirus Protection To Your Windows, Mac Or Mobile Devices. Check out this Post to Get All Information About How You Can Redeem Your Norton Product Key and Get You Antivirus Activated In Few Easy steps.

    norton.com/setup
    norton.com/setup

    ReplyDelete
  9. Best Printer Repair service with iYogi. We assist all printers like Epson, Canon, HP, and Brother with driving force's installation, printer networking settings and attach printer mistakes over the phone.
    epson printer offline

    ReplyDelete

  10. Downloading process of the Office setup starts now. How to install Microsoft Office Setup? There are two methods to install Office setup in your PC - through a CD and by downloading. When you buy Office setup offline, then you have to install through a CD, and you get the setup file for installation if you get the setup online.

    Office.com/Setup

    ReplyDelete


  11. the best site for Satta king, leak number & all record charts.We provide 100% fix number from direct Satta company which includes all famous games like Desawar, Gali Satta, Ghaziabad, Faridabad and other games of Satta Market(Satta matka) is also a simple game and essentially is a form of old lottery games.

    satta matka
    satta
    satta king
    satta matka results

    fmovies

    ReplyDelete
  12. These days users, both individuals as well as business owners are highly concerned about computer security. If one safeguards their computer with antivirus, then they can protect their devices from viruses and other malware. For making the device secure, one needs to download Norton security software. To set up the Norton antivirus, the users need to go through its procedure of downloading, installing and activating on www.norton.com/setup.

    ReplyDelete
  13. https://www.fitdiettrends.com/ultra-cbd-extract-au/
    http://fit-diet-trends.over-blog.com/2019/10/ultra-cbd-extract
    https://www.youtube.com/watch?v=Qf0mC2BZ_Pc
    https://sites.google.com/site/fitdiettrends/ultra-cbd-extract
    https://soundcloud.com/fit-diet-trends/ultra-cbd-extract
    https://fitdiettrends.tumblr.com/post/188336395083/ultra-cbd-extract
    https://fitdiettrends.wordpress.com/2019/10/14/ultra-cbd-extract/

    ReplyDelete
  14. the user must purchase a license of Avg antivirus from the retail store or online. Though download AVG software is the easiest process, your system must match the essential prerequisites of Avg antivirus. Once you will get AVG activation code and will create an AVG account , then you can install AVG with license number. install-avg-with-license-number | www.avg.com/activation | www.avg.com/retail

    ReplyDelete

  15. I’m extremely affected regarding the information you offer in your articles. i need to say am extremely overpowered by your whole story. It’s tasking to induce such quality data on-line these days. I expect to staying here for an extended time.
    norton.com/setup | Office.com/setup | norton.com/setup

    ReplyDelete
  16. Awesome! Never seen so nice post. Keep going. You are the best blogger!

    Visit: www.trendmicro.com/bestbuypc

    ReplyDelete
  17. It is very helpful to secure your device and it is very light weighted antivirus product.

    Visit: Enter Norton product key

    ReplyDelete
  18. Wonderful blog post, thank you so much for the great information which you provided.

    Visit: www.norton.com/setup

    ReplyDelete
  19. Wow!! It's a really great experience sharing with us. I like your post, it's a really interesting.

    Visit: www.norton.com/setup

    ReplyDelete
  20. The hp123 is actually a website which is set up by HP technicians to help people set up their hp123.com/setup Printer by downloading the printer drivers and other software through this website.
    hp123.com/setup |
    123.hp.com/setup
    hp123.com/setup |
    123.hp.com/setup
    hp123.com/setup |
    123.hp.com/setup

    ReplyDelete
  21. Canon printer that can be downloaded via canon.com/ijsetup page is the best wireless printer that you can connect to your device and print data smoothly. Canon ijsetup CD is not the well-suited technique to use canon.comijsetup setup installation for longer.
    canon.com/ijsetup |
    canon ij setup |
    canon ij printer setup |

    ReplyDelete

  22. Webroot antivirus software is good and did well in our ratings. It offers basic protection at a low price, which is great for some users. ... If you're looking to save money on antivirus software and need standard features like cloud storage and a password manager, Webroot should be on your short list. webroot.com/safe | www.webroot.com/safe | webroot.com/secure | webroot.com/safe | www.webroot.com/safe | webroot.com/secure | webroot.com/secure


    ReplyDelete
  23. Canon.Com/ijsetup will manual you to Install Canon printer brand new updated drivers, for Canon printer setup you could additionally visit ij.start canon. If we talk about printers the first name comes in our thoughts is ij.start.canon printer, on this internet site we will inform you of a way to setup deploy your canon printer with little information about computers. ij.start canon | ij.start.canon | ij.start canon | ij.start.canon
    ij.start canon | ij.start.canon | Canon.com/ijsetup

    ReplyDelete

  24. https //my.norton.com/home/setup - Norton Internet Security or Norton Antivirus products are the essential tools for protecting the computer or digital devices from malware, spyware, Trojans and other virus attacks. For Internet threats Norton products are very effective. Though the installation process of norton.com/setup product is easy yet for non-tech people or beginners find some hindrances with the installing procedure. To give you a solution, our tech support team will help you and fix your problem with ease.

    ReplyDelete
  25. WOW! I Love it...
    and i thing thats good for you >>


    MOVIE TRAILER F9 Fast and Furious Thank you!

    ReplyDelete
  26. Webroot antivirus is one in all the simplest code to safeguard all digital devices one will get additional details on www.webroot.com secureanywhere. Visit: Webroot geek squad

    ReplyDelete
  27. Thanks for sharing the information. Your blog has always been a source of great tips.
    www.norton.com/setup

    ReplyDelete
  28. Nice blog, hope you are writing the same in future.

    Visit: www.norton.com/setup

    ReplyDelete
  29. This post is very informative on this topic Thank you for sharing this post with us.

    Visit: www.trendmicro.com/bestbuypc

    ReplyDelete
  30. This post is very informative on this topic Thank you for sharing this post with us.
    Visit: www.trendmicro.com/downloadme

    ReplyDelete
  31. comcast email sign in | Xfinity Email Sign in – Comcast now Xfinity is a large US based internet provider offering a wide range of communication services. TV, cable internet, Comcast Email and voicemail are some of the available service.

    ReplyDelete
  32. I wish someday I would be able to create on such write up

    yellowatone hoodie coat



    ReplyDelete

  33. very interesting , good job and thanks for sharing such a good blog.
    www.norton.com/setup

    ReplyDelete
  34. Nice one! thank you so much! Thank you for sharing this post. Your blog posts are more interesting and impressive.
    comcast email sign in

    ReplyDelete
  35. I check your blog every day and would try to find some of your blog site. Thank you and wait for your new article.

    These printers are the best printer to use offline mode and give us the best results and improved print. So I suggest everyone WIFI printer. canon.com/ijsetup the best WiFi printer, it can be your router or hotspot Print from PC, Mac and print from Android. So I would like to buy online at the best rates this printer.

    ReplyDelete
  36. :Canon offers the Printer Setup download link where you can install the printer setup further Canon.com/ijsetup

    ReplyDelete
  37. Download the latest hp printer drivers ,set up the hp Officejet printer and get started with your new

    printer[url=https://oj3830-oj3830.com/123-hp-com-dj3630/][b]123.hp.com/dj3630[/b][/url][b]|[/b][url=https://oj3830-oj3830.com/123-hp-com-dj3752/][b]123.Hp.Com/dj3752[/b][/url][b] | [/b][url=https://oj3830-oj3830.com/123-hp-com-dj2652/][b]123.Hp.Com/dj2652[/b][/url]

    ReplyDelete
  38. www.webroot.com/safe is the strong antivirus which detects the cyber threat immediately and blocks it, before it harms your gadget. You can install this antivirus software through.

    ReplyDelete
  39. webroot.com/safeantivirus is an amazing software that keeps eyes on every doubtful thing in your device and if it’s a thumb down, Webroot wipes out the attacker and reverse its action. Webroot does its job very well. It even wipes out the virus and blocks the website which contains the virus.

    ReplyDelete
  40. The Product key is 25 alphanumeric characters license code used to activate Office com/setup Home & Student 2019 available to purchase online as well on retail stores. Microsoft covers the code with scratch-proof covering for privacy and security.
    office.com/setup home & student 2019

    ReplyDelete
  41. thanks for telling about us on the designing, several ways in creating an approach to and then is keeping with soft hands when and it by clicking a and when this goes to do it all form. click for more information to prints one of the things. the work process to do. yeah, that's the thing that is so so good to get any of them here the user should do what he wants.
    cannon pixma 470

    ReplyDelete
  42. When you do connect the Canon printer into your pc, your system does not need to install the driver on it. https //ij.start.canon , http //ij.start.canon .

    ReplyDelete
  43. This comment has been removed by the author.

    ReplyDelete
  44. To download and install the Microsoft Office setup on your computer, you should have t
    he Office product key Office.com/setup Home & Student 2019 , Office.com/setup Home Student 2019 , www.office.com/setup Home Student 2019 .

    ReplyDelete
  45. webroot.com/secure Antivirus with Spy Sweeper, previously known as Webroot AntiVirus with AntiSpyware, is an antimalware utility developed by webroot.com/secure. The product added an antivirus detection engine to the company’s previous standalone anti spyware product Spy Sweeper. 123.hp.com/laserjet

    ReplyDelete
  46. Microsoft 365 plans for personal and home provides robust Office desktop apps including Word, PowerPoint, Excel, Outlook, and OneNote.
    microsoft365.com/setup | microsoft365.com/setup |
    microsoft365.com/setup | microsoft365.com/setup


    ReplyDelete
  47. Thanks for sharing this informative blog, hard to find informative content.

    The SalezShark
    Texas CRM software
    works together repository to conduct your sales, marketing, and customer support activities cooperatively to streamline your sales process, and customers in one platform.

    ReplyDelete
  48. Excellent Blog! I would like to thank for the efforts you have made in writing this post. I am hoping the same best work from you in the future as well. I wanted to thank you for this websites! Thanks for sharing.

    Canon to use the printer service call, you can be set to download can get the full set of Canon printer and Canon printer setup driver Canon printer. Bellows steps will help you set up a Canon wireless printers with the help of Canon.

    Click Here: how to override printer ink levels canon how to override printer ink levels canon

    ReplyDelete
  49. Go through the canon.com/ijsetup webpage, click Set Up and start the Canon ij printer setup process. If you have a Canon inkjet printer (wireless or wired), setting up the complete printer is easy. To download canon printer drivers and install them on your pc, visitcanon.com/ijsetup
    Go to norton.com/setup and download Norton setup.
    norton.con/setup or norton.com/setup

    ReplyDelete
  50. Open officesetup on home page, and click & Install Office& or Office 356 apps option to begin
    office.com/setup home & student 2019
    ,Paypal Login , www.trendmicro.com/activate , norton.com/setup .



    ReplyDelete
  51. Office setup is the collection of software used by almost every kind of individuals including companies, students & professionals. The first version of Microsoft office was introduced with Microsoft Office, Microsoft Excel, and Microsoft PowerPoint. Microsoft Word was developed to create documents.

    office.com/setup home & business 2019
    office setup home & business 2019

    ReplyDelete
  52. Thank you for sharing those updates with us! I hope that you’ll make some other notes of this sort, as well.
    When you purchase a new printer, it delivers good results and works smoothly. But, as time passes by, it may give you the worst experience. Sometimes, you may display specific error messages that are enough to trouble you in your work, and other times it may undergo some technical difficulty. Although there are several reasons behind it one of the most common reasons is that your printer and the device from which you are giving commands to it are unable to establish a connection. Get complete solutions for your printer and other smart devices. We will guide you in best of our knowledge. http //ij.start.canon setup

    ReplyDelete
  53. Welcome to the HP site. Download the software SW from hp setup and set up your printer. Then, Get started to installed hp setup and get able to connect the printer to the network through a USB cable or use a wireless HP printer.
    123.hp.com/setup
    norton.com/setup
    http://ij.start.canon
    https://ij.start.canon

    ReplyDelete
  54. Next time whenever you'll go to microsoft365.com/setup you won't require to enter the product key. As the product key is activated, simply enter the Microsoft login account to install Microsoft 365 apps. microsoft365.com/setup , All your transactions are secured with a PayPal login. Paypal Login | Paypal Login

    ReplyDelete
  55. There probably been numerous challenges in giving this data, in any case, a debt of gratitude is in order for giving.
    If you want to take print then there is a good chance for you why our printer will get very good facilities and our website will also tell you how to use it. And you will get a lot of information, canon.comijsetup you must try once and comment your questions to us. Thank you

    ReplyDelete
  56. This comment has been removed by the author.

    ReplyDelete
  57. We have brought on my http//ij.start.canon site for you high-speed and low-maintenance commercial inkjet multi-functional printers. If anyone wants to buy this printer, get easily through my site.

    ReplyDelete
  58. Camps Intuit is the portal that provides you the QuickBooks business accounting and management software available.
    camps.intuit.com |
    camps.intuit.com

    ReplyDelete
  59. To activate Trend micro, make sure you already have an activation code that you’ll probably enter on the
    [url=https://www.trendmicrocomactivate.com/]www.trendmicro.com/activate[/url]

    ReplyDelete
  60. Defend against the unknown threats and proceed for Trend micro download with www.trendmicro.com/activate having Advanced Machine Learning Technology.If you have a Norton product key norton.com/setup and download Norton setup. Go through the 123.hp.com/laserjet | paypal login provides wired and wireless. microsoft365.com/setup

    ReplyDelete
  61. The Webroot program is highly rated software to protect your devices & data available for download at webroot.com/safe.
    123.hp.com/setup |
    ij.start.canon|
    a>

    ReplyDelete
  62. Excellent and nice post.
    Our site provides full information about canon printers. If you facing some difficulties in using canon printer, Just give your printer modal number, our technical support team will assist you with online chat or on call.
    http //ij.start.canon setup

    ReplyDelete
  63. The Webroot program is highly rated software to protect your devices & data available for download at webroot.com/safe.
    123.hp.com/setup |
    ij.start.canon| ij.start.cannon/ts3322 |

    ReplyDelete

  64. แทงบอล 168

    สนุกสุดก็เว็บนี้

    ReplyDelete
  65. https://ofoghnews.ir/344928/%D8%AE%D8%B1%DB%8C%D8%AF-%D9%85%D9%88%D8%A8%D8%A7%DB%8C%D9%84-%D8%AF%D8%B1-%DA%86%D9%86%D8%AF-%DA%AF%D8%A7%D9%85-%D8%B3%D8%A7%D8%AF%D9%87-%D9%88-%DA%A9%D8%A7%D8%B1%D8%A8%D8%B1%D8%AF%DB%8C/Mobile shopping is one of the challenges that can not be easily overcome. Among the many mobile phone models on the market, it is really difficult to choose one

    The higher the budget, the harder it is to make the purchase. To select and buy the best mobile phone on the market for the desired budget, we have designed a few steps that you can use to find your ideal model more easily and quickly. The best mobile phones are always out of sight of users, but this way you can buy more easily. Of course, to introduce these solutions for buying mobile phones, we have also helped one of the consultants and experts in this category on the mobile help site, so that the opinions provided are as expert as possible.

    ReplyDelete
  66. The process to set up PIXMA ts3122 starts from ij.start.canon/ts3122 and download the latest full driver & software package for PIXMA TS3122. You can learn to set up this multifunction printer model. Instructions on this page include everything from printer unpacking, installation, configuration, WiFi network establishment to complete canon printer drivers setup.The setup process for every Canon model is almost similar, however the download through https //ij.start.cannon or http //ij.start.cannon and installation process may differ. Let’s get started.ij.start canon is the manufacturer site to download Canon printer drivers. Install and set up Canon Printer from ij.start.canon | canon.com/ijsetup | cannon/ijsetup | Cox High-Speed Internet provides its users with COX Communications..Cox webmail Login

    ReplyDelete

  67. Its a really interesting and informative article for me. I appreciate your work and skills. Lucifer Rising Jacket

    ReplyDelete
  68. Let’s get started.ij.start canon is the manufacturer site to download Canon printer drivers. Install and set up Canon Printer from ij.start.canon

    ReplyDelete
  69. Where to buy game boards? Every purchase needs reviews. For example, to buy fruit, you should check that the fruits are healthy? Or does the fruit seller sell quality fruits? Is the price of fruit reasonable? However, in a face-to-face purchase, these reviews are not very difficult. But when shopping online, you should make sure that in addition to the right price and quality, you get after-sales service and quality guarantee.
    https://face3.ir/bazi/

    ReplyDelete
  70. The setup process for every Canon model is almost similar, however the download through https //ij.start.cannon or http //ij.start.cannon and installation process may differ.Https //ij.start.cannon Depending on your requirement, it offers a type printer including PIXMA, SELPHY, MAXIFY, etc. canon.com/ijsetup Some factors need to be in mind while choosing an inkjet printer for you. Later, you can easily set up your Canon printer through drivers from ij.start.cannon wireless connection, USB, and a few components.

    ReplyDelete
  71. The price air conditioner price in pakistan is quite broad, catering to various segments of the market. Starting from basic models that can cost around PKR 40,000, the prices can go up to PKR 200,000 or more for premium units featuring the latest inverter technology, energy efficiency, and smart controls

    ReplyDelete
  72. The air conditioner price in pakistan is quite broad, catering to various segments of the market. Starting from basic models that can cost around PKR 40,000, the prices can go up to PKR 200,000 or more for premium units featuring the latest inverter technology, energy efficiency, and smart controls

    ReplyDelete
  73. The adoption of advanced encryption methods is a testament to Indibet commitment to data protection

    ReplyDelete
  74. Protein is essential for maintaining and growing muscle. Protein powder is a convenient and quickly absorbed kind of protein that is great for post-workout nourishment, even though complete food sources such as fish, poultry, eggs, and beans are all excellent sources. Here are some advantages of protein powder. KNOW MORE : rc pro antium

    ReplyDelete