Friday, September 23, 2011

Managing Decoupling Part 4 -- The ID Lookup Table

Today I am going to dig deeper into an important and versatile data structure that pops up all the time in the BitSquid engine -- the ID lookup table.

I have already talked about the advantages of using IDs to refer to objects owned by other systems, but let me just quickly recap.

IDs are better than direct pointers because we don’t get dangling references if the other system decides that the object needs to be destroyed.

IDs are better than shared_ptr<> and weak_ptr<> because it allows the other system to reorganize its objects in memory, delete them at will and doesn’t require thread synchronization to maintain a reference count. They are also POD (plain old data) structures, so they can be copied and moved in memory freely, passed back and forth between C++ and Lua, etc.

By an ID I simply mean an opaque data structure of n bits. It has no particular meaning to us, we just use it to refer to an object. The system provides the mechanism for looking up an object based on it. Since we seldom create more than 4 billion objects, 32 bits is usually enough for the ID, so we can just use a standard integer. If a system needs a lot of objects, we can go to 64 bits.

In this post I’m going to look at what data structures a system might use to do the lookup from ID to system object. There are some requirements that such data structures need to fulfill:

  • There should be a 1-1 mapping between live objects and IDs.
  • If the system is supplied with an ID to an old object, it should be able to detect that the object is no longer alive.
  • Lookup from ID to object should be very fast (this is the most common operation).
  • Adding and removing objects should be fast.

Let’s look at three different ways of implementing this data structure, with increasing degrees of sophistication.

The STL Method


The by-the-book object oriented approach is to allocate objects on the heap and use a std::map to map from ID to object.

typedef unsigned ID;

struct System
{
	ID _next_id;
	std::map<ID, Object *> _objects;

	System() {_next_id = 0;}

	inline bool has(ID id) {
		return _objects.count(id) > 0;
	}
	
	inline Object &lookup(ID id) {
		return *_objects[id];
	}
	
	inline ID add() {
		ID id = _next_id++;
		Object *o = new Object();
		o->id = id;
		_objects[id] = o;
		return id;
	}
	
	inline void remove(ID id) {
		Object &o = lookup(id);
		_objects.erase(id);
		delete &o;
	}
};

Note that if we create more than four billion objects, the _next_id counter will wrap around and we risk getting two objects with the same ID.

Apart from that, the only problem with this solution is that it is really inefficient. All objects are allocated individually on the heap, which gives bad cache behavior and the map lookup results in tree walking which is also bad for the cache. We can switch the map to a hash_map for slightly better performance, but that still leaves a lot of unnecessary pointer chasing.

Array With Holes


What we really want to do is to store our objects linearly in memory, because that will give us the best possible cache behavior. We can either use a fixed size array Object[MAX_SIZE] if we know the maximum number of objects that will ever be used, or we can be more flexible and use a std::vector.

Note: If you care about performance and use std::vector<T> you should make a variant of it (call it array<T> for example) that doesn’t call constructors or initializes memory. Use that for simple types, when you don’t care about initialization. A dynamic vector<T> buffer that grows and shrinks a lot can spend a huge amount of time doing completely unnecessary constructor calls.

To find an object in the array, we need to know its index. But just using the index as ID is not enough, because the object might have been destroyed and a new object might have been created at the same index. To check for that, we also need an id value, as before. So we make the ID type a combination of both:

struct ID {
	unsigned index;
	unsigned inner_id;
};

Now we can use the index to quickly look up the object and the inner_id to verify its identity.

Since the object index is stored in the ID which is exposed externally, once an object has been created it cannot move. When objects are deleted they will leave holes in the array.


When we create new objects we don’t just want to add them to the end of the array. We want to make sure that we fill the holes in the array first.

The standard way of doing that is with a free list. We store a pointer to the first hole in a variable. In each hole we store a pointer to the next hole. These pointers thus form a linked list that enumerates all the holes.


An interesting thing to note is that we usually don’t need to allocate any memory for these pointers. Since the pointers are only used for holes (i. e. dead objects) we can reuse the objects’ own memory for storing them. The objects don’t need that memory, since they are dead.

Here is an implementation. For clarity, I have used an explicit member next in the object for the free list rather than reusing the object’s memory:

struct System
{
	unsigned _next_inner_id;
	std::vector<Object> _objects;
	unsigned _freelist;

	System() {
		_next_inner_id = 0;
		_freelist = UINT_MAX;
	}

	inline bool has(ID id) {
		return _objects[id.index].id.inner_id == id.inner_id;
	}
	
	inline Object &lookup(ID id) {
		return _objects[id.index];
	}
	
	inline ID add() {
		ID id;
		id.inner_id = _next_inner_id++;
		if (_freelist == UINT_MAX) {
			Object o;
			id.index = _objects.size();
			o.id = id;
			_objects.push_back(o);
		} else {
			id.index = _freelist;
			_freelist = _objects[_freelist].next;
		}
		return id;
	}
	
	inline void remove(ID id) {
		Object &o = lookup(id);
		o.id.inner_id = UINT_MAX;
		o.next = _freelist;
		_freelist = id.index;
	}
};

This is a lot better than the STL solution. Insertion and removal is O(1). Lookup is just array indexing, which means it is very fast. In a quick-and-dirty-don’t-take-it-too-seriously test this was 40 times faster than the STL solution. In real-life it all depends on the actual usage patterns, of course.

The only part of this solution that is not an improvement over the STL version is that our ID structs have increased from 32 to 64 bits.

There are things that can be done about that. For example, if you never have more than 64 K objects live at the same time, you can get by with 16 bits for the index, which leaves 16 bits for the inner_id. Note that the inner_id doesn’t have to be globally unique, it is enough if it is unique for that index slot. So a 16 bit inner_id is fine if we never create more than 64 K objects in the same index slot.

If you want to go down that road you probably want to change the implementation of the free list slightly. The code above uses a standard free list implementation that acts as a LIFO stack. This means that if you create and delete objects in quick succession they will all be assigned to the same index slot which means you quickly run out of inner_ids for that slot. To prevent that, you want to make sure that you always have a certain number of elements in the free list (allocate more if you run low) and rewrite it as a FIFO. If you always have N free objects and use a FIFO free list, then you are guaranteed that you won’t see an inner_id collision until you have created at least N * 64 K objects.

Of course you can slice and dice the 32 bits in other ways if you hare different limits on the maximum number of objects. You have to crunch the numbers for your particular case to see if you can get by with a 32 bit ID.

Packed Array


One drawback with the approach sketched above is that since the index is exposed externally, the system cannot reorganize its objects in memory for maximum performance.

The holes are especially troubling. At some point the system probably wants to loop over all its objects and update them. If the object array is nearly full, no problem, But if the array has 50 % objects and 50 % holes, the loop will touch twice as much memory as necessary. That seems suboptimal.

We can get rid of that by introducing an extra level of indirection, where the IDs point to an array of indices that points to the objects themselves:


This means that we pay the cost of an extra array lookup whenever we resolve the ID. On the other hand, the system objects are packed tight in memory which means that they can be updated more efficiently. Note that the system update doesn’t have to touch or care about the index array. Whether this is a net win depends on how the system is used, but my guess is that in most cases more items are touched internally than are referenced externally.

To remove an object with this solution we use the standard trick of swapping it with the last item in the array. Then we update the index so that it points to the new location of the swapped object.

Here is an implementation. To keep things interesting, this time with a fixed array size, a 32 bit ID and a FIFO free list.

typedef unsigned ID;

#define MAX_OBJECTS 64*1024
#define INDEX_MASK 0xffff
#define NEW_OBJECT_ID_ADD 0x10000

struct Index {
	ID id;
	unsigned short index;
	unsigned short next;
};

struct System
{
	unsigned _num_objects;
	Object _objects[MAX_OBJECTS];
	Index _indices[MAX_OBJECTS];
	unsigned short _freelist_enqueue;
	unsigned short _freelist_dequeue;

	System() {
		_num_objects = 0;
		for (unsigned i=0; i<MAX_OBJECTS; ++i) {
			_indices[i].id = i;
			_indices[i].next = i+1;
		}
		_freelist_dequeue = 0;
		_freelist_enqueue = MAX_OBJECTS-1;
	}

	inline bool has(ID id) {
		Index &in = _indices[id & INDEX_MASK];
		return in.id == id && in.index != USHRT_MAX;
	}
	
	inline Object &lookup(ID id) {
		return _objects[_indices[id & INDEX_MASK].index];
	}
	
	inline ID add() {
		Index &in = _indices[_freelist_dequeue];
		_freelist_dequeue = in.next;
		in.id += NEW_OBJECT_ID_ADD;
		in.index = _num_objects++;
		Object &o = _objects[in.index];
		o.id = in.id;
		return o.id;
	}
	
	inline void remove(ID id) {
		Index &in = _indices[id & INDEX_MASK];
		
		Object &o = _objects[in.index];
		o = _objects[--_num_objects];
		_indices[o.id & INDEX_MASK].index = in.index;
		
		in.index = USHRT_MAX;
		_indices[_freelist_enqueue].next = id & INDEX_MASK;
		_freelist_enqueue = id & INDEX_MASK;
	}
};

97 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. I don't really see the point of your suggestion. What you are returning externally is a pointer to an object member. For that to remain valid, the object cannot be moved or deleted. But it seems in that case, you might just as well return a pointer to the object itself.

    ReplyDelete
  5. This comment has been removed by the author.

    ReplyDelete
  6. This comment has been removed by the author.

    ReplyDelete
  7. By using the described method in this article:

    1) How are the systems only coupled on a higher level if they for example share the same transforms? I am not sure if in your examples you are coupling from inside the systems (lower level) or from a higher level. Currently I do this by feeding the systems batches of arrays (transforms, bodies, render properties) from an higher level. Multiple systems may share the same arrays but internally keep them in a different order. Is that what you are doing?

    2) How do the systems share the same transforms but access them in a different order while keeping all access patterns sequential? For ex. the renderer needs to access the same transforms in texture order, the physics engine needs to access the same transforms in proximity order. Currently I do this by sorted lists for the renderer (initial radix sort, then merge sort every several frames) and keeping separate lists for physics (for each hashgrid cells).

    3) When an object/entity gets deleted or a component from an object/entity is getting removed: How are you finally removing the holes? Are you swapping out one element every frame? How are you skipping the holes when sequentially processing the arrays. Are you using the index array to only process what is needed? I am not sure if this is what you are already saying in the article.

    Sorry for the many questions and thank you for this great article.

    ReplyDelete
  8. Forgot to ask one last question:

    4) Are you only keeping a ID lookup array per system or do you also have a high level lookup array of some sort?

    Actually. Asking all those questions are helping me to anwer them myself :P. Of course I am still curious about your anwers.

    ReplyDelete
  9. 1) Usually by feeding, the higher level system extracts a list of transforms from one low level system and sends them to another low level system. Both systems keep there own local lists of transform.

    2) Each system has its own list of data that it needs to "crunch" on. So the animation system, the scene graph and the renderer all have separate lists of object transforms, each sorted according to that system's preferences.

    3) With the last solution, sequential processing is only done on the "object array", not on the "index array", which means there are no holes.

    4) Each system keeps its own lookup table... there is no master lookup table.

    ReplyDelete
  10. @Niklas. Thank you for the answers. I was a bit confused by the terminology used, so I understand the part about the lookup arrays now. Sorry for that ;)

    About the answer on the 1st question. How exactly is the higher level system feeding the transforms from one low level system to the other? Are you doing this every frame? As I can imagine that could potentially cause each low level system to constantly (each frame) re-sort its list of transforms to its preferred order (for example when every transform was changed by the physics system and is fed into another system which requires another order).

    What I am currently doing is keeping the list of transforms the same for most systems so I can pass them around without re-sorting. The only system that will re-sort the list of transforms, is the render system. Can you advice me a better way?

    ReplyDelete
  11. An example...the animation system has a list of bone transforms

    B_1 B_2 B_3 ...

    The scenegraph has a list of node transforms

    N_1 N_2 N_3 ...

    Some of the nodes correspond to bones, but there are also some nodes that don't have corresponding bones.

    A higher level system AnimationToSceneGraph knows about both systems and the connection between nodes and bones. Each update it does (in pseudocode):

    for (i,tm) in Bones:
    Node[bone_to_node[i]].tm = tm

    As you see this does not cause any list resorting. There is some extra copying going on, that you wouldn't get if you shared the data, but I think that is outweighed by the fact that each system can organize its own data so that it can process it as fast as possible.

    ReplyDelete
  12. Hoi Niklas. How do you manage one-to-many relations between the index array and the object array. In case of your bone transform example: an entity having multiple bones?

    ReplyDelete
  13. There are no one-to-many relationships here. An ID is a way for an external system to refer to a single object. If the external system needs to refer to more than one object, it can keep a list of IDs.

    ReplyDelete
  14. Niklas, correct me please if I'm wrong, but there seems to be a small bug in your packed array implementation. Steps to reproduce:

    1) Add MAX_OBJECTS number of objects without any removals. After adding the last one _freelist_dequeue will be equal MAX_OBJECTS.
    2) Remove any object
    3) Add a new object. Since _freelist_dequeue still points MAX_OBJECTS a memory corruption will occur

    I guess, the fix can be is to add the following to the end of the remove() method:

    if(_freelist_dequeue == MAX_OBJECTS)
    _freelist_dequeue = _freelist_enqueue;

    ReplyDelete
  15. You are right. The code as written only supports a maximum of MAX_OBJECTS-1 objects.

    ReplyDelete
  16. This comment has been removed by the author.

    ReplyDelete
  17. This comment has been removed by the author.

    ReplyDelete
  18. How do you handle the case there a lower system resorts a list, so that the higher system still have the right indices?

    ReplyDelete
  19. The ID doesn't have to be an index directly into the lower systems list. There can be a lookup array that offers a level of indirection. That allows for resorting.

    ReplyDelete
  20. Hi, there is a part in your packed array code I don't understand. What is the point of doing this :
    in.id += NEW_OBJECT_ID_ADD;
    ?

    ReplyDelete
  21. To ensure that each object gets a new unique ID, so that you can call has() on an old object ID to check whether that object still exists or not.

    ReplyDelete
  22. Currently trying to grasp those (that I find great) design ideas of DoD and back-to-C I still have trouble wrapping my head around something.

    When you do (in your example of October 10, 2011 in the comments here):

    for (i,tm) in Bones:
    Node[bone_to_node[i]].tm = tm

    Doesn't that make random access through the Node array completely defeating the cache access? Granted you will have at some point to do it, and here it's the Bone that update the node. But what when it's the other way around? A renderer, audio source, script needing the node transform? Wouldn't it be faster to have each renderer stocking the id of the corresponding node, and grab the transform during the update of the RendererManager? Instead of copying it and then iterating over the array, because in most case, you will only need it once for each object update.

    I just can't manage to find a design where I won't have to do those random array access during the update anyway, loosing the point of having my data well contiguous in memory, because I always need to do random access for each of them each update, will it be in a pre-fetch like your exemple, or in the loop iterating over each of them to update them...

    ReplyDelete
  23. It is not complete random access, because you could take steps to ensure that you bone list is sorted in the same order as the node list and then it would still be in-order access (just skipping the nodes that don't have associated bones).

    Even if you don't do that, random access in a small contiguous memory area (the node matrix list) is still a lot better than random access all over memory, because you will get fewer L2 cache misses.

    ReplyDelete
  24. For fun and new ideas: Interface implementation in Go
    http://research.swtch.com/interfaces

    ReplyDelete
  25. After having read this and the previous three articles in this series (And that's all I've read of your blog so far), it seems like you are really comfortable working with raw memory, bit manipulation, etc. Do you have any recommended readings regarding any of those things; books, articles, entire websites, or anything comprehensive? I searched quite a bit for a book that covers data oriented design and things like that but maybe I am searching for the wrong things.
    I guess I should also ask: Have you just collected and applied things from many different sources and from having a good background in general computer science? I would very much like to have the understanding that you have when you created these neat and intelligent solutions.

    ReplyDelete
    Replies
    1. For me it's been a journey over time... seeing that something is slow in C++, understanding why, stepping through code... (seeing what actually happens when you copy a nested STL structure, or call atof() on windows can be eye-opening) then thinking about how things can be done better, getting experience with doing those kinds of designs, seeing what other people do, etc.

      Good things to try:

      * Read some C books. C books usually focus more on what actually happens in memory than C++ that tends to gloss over things.

      * Good code to try to read and understand:

      - Container class implementations (std::vector, etc).

      - Memory allocators (dlmalloc)

      - Script language implementations (luajit)

      * Now try and write some of those things on your own.

      Delete
    2. Thank you very much for your reply and your time. I think you've given me some very good advice. I'll be reading some C books and studying some containers, allocators, and everything else that might do something interesting or useful with memory.

      Delete
  26. Your Object has id field. Wouldn't it be better to separate the indices in two separate arrays that point one to another, practically creating sparse set (http://research.swtch.com/sparse)?

    ReplyDelete
    Replies
    1. good eye ... when i first read about spare sets, my first problem was: we have dense, we sparse arrays of integers, but where the heck i store my actual data? I guess OP spares extra array by introducing the ID in the actual data, Objects

      Delete
  27. To avoid holes, you can just swap the undesired element with the last inserted element. Update index then.

    ReplyDelete
  28. This comment has been removed by the author.

    ReplyDelete
  29. In the event that you're battling with Epson wf-3640 printer disconnected blunders code and neglecting to pass on your printer returned into the online mode, you can contact with our gifted specialists to fix these sort of slip-ups with flawlessness. However, before reaching us, you can go to directly here
    epson printer setup

    ReplyDelete
  30. Hello! My name is Andy Nickerson, a professional technical writer written immense blogs on Epson printer related to its problematic issues and top-notch troubleshooting procedures. Therefore, you are not needed to feel helpless also even feel alone in this technical world,There are many supporting candidates who provide right solution. Rather than wandering for knowing the solution of the problem Epson printer in error state lend an instant hand with me. I and my experienced squad is working full time only for your help. Hence, dial toll-free provided number and make one-to-one conversation for reliable and cost-effective solution.

    ReplyDelete
  31. Thanks for sharing this post , We work independently as a third party tech support provider for brother printer users. We provide 24/7 live support or help for brother printer users in very nominal charges. If you’re facing error code 2147500037, you can call our trained printer experts.

    For printer setup :- brother printer setup

    printer offline issue :- Brother printer offline

    ReplyDelete
  32. Hii Guys hope everybody is doing great! I am here to ask you the reason for the norton.com/setup. Actually, I have been frustrated with this setup and tried my best to install this antivirus but couldn’t install it. Do anyone of you know what the reason for this antivirus is and how to install it. Guys if you know the reasons because tell me in detail. I will be grateful for your assistance. Waiting for your response!

    ReplyDelete
  33. If you are a new user of HP printer and want to setup appropriately, then visit the link
    123.hp.com/setup and follow-up the instructions. Once you successfully setup your HP printer, you can effortlessly print, scan or fax. Setup process consists of a printer driver to install, load a stack of white paper into the input tray, installing the ink cartridge, network connection, downloading the printer software on system and many more.

    ReplyDelete
  34. Thanks for sharing useful information Brother Printer can’t connect to wifi, which is one of the most common problems nowadays. Every day there are millions of people complaining about this kind of issue.

    ReplyDelete
  35. Useful information ..I am very happy to read this article..thanks for giving us this useful information. Fantastic walk-through. I appreciate this post.
    Windows update error 80246007

    ReplyDelete
  36. Thanks for providing this information. If you are looking for fast assistance for yahoo mail then immediately connect Yahoo customer service team for the best solution.

    ReplyDelete
  37. Nowadays many apps are allowing the users to enable the dark theme. Also, Snapchat dark mode is among those with fascinating features. Here we discuss easy and best steps to enable the dark mode in Snapchat - Snapchat Dark Mode

    ReplyDelete
  38. Very nice article, I found it very useful .Even I have this wonderful website, Good quality of windows and doors are very essential for any workplace.
    Epson printer offline

    ReplyDelete
  39. Epson printer is better known for its exceptional user-friendly features. Yet, in many instances, especially new users do not know how to setup Epson printer to a Wi-Fi network. If you are facing the same difficulties to setup your Epson printer to a wireless network, get connect with our Printer expert team straight away.

    ReplyDelete
  40. Thanks for the informative and helpful post, obviously in your blog everything is good.
    Usb Device Not Recognized Windows 10

    ReplyDelete
  41. Setting up an email in QuickBooks is an easy process one can do, however, some users are not able to do it one their own. If you do not know how to setup email in QuickBooks in your system, you must get connected with our tech team to fetch email in your software instantly.

    ReplyDelete
  42. Data Recovery Service Center - Virus Solution Provider
    Data Recovery Customer Care We at Virusolutionprovider, understand the vital importance of your data and its significance in your business. We help you retrieve and recover your lost data due to any technical glitch or human error. Our programs are specially designed to scan whole memory hierarchy for lost data files and to retrieve the lost data back to the initial storage location. Our aim is to retrieve all of your data without any data or information loss. We have a skilled team with years of experience in the field of data recovery. We are committed to provide effective solutions related to data loss to our customers, with minimum response time and at optimal price.
    Go to the Official Website
    https://www.virusolutionprovider.com
    Call our Support Number = +91 (999) 081-5450 (WhatsAap call or Direct Call)

    ReplyDelete
  43. Forgot Yahoo Email Password is not the big thing. Yahoo also provides a password recovery option. Still, some users face issues while proceeding on the process in order to recover the forgotten Yahoo account password. If you are getting similar issue, you must know that you are just little bit away to get instant help.

    ReplyDelete
  44. Yahoo mail is no doubt one of the tough competitors against all other popular webmail service providers. But some of its users are complaining about the issue of Yahoo mail not sending emails. If you are too resentful about the same issue, get connect with our Yahoo executive team. You can avail of our Skype facility to connect with one of our experts.

    ReplyDelete
  45. Choose Assignment Help when you have no one to ask your concerns while studying at American universities. Students can connect with professional academic writers using online assignment writing services and discuss their concerns even staying at home. You don’t need to make any physical contact with anyone when you have the option of online services.
    help with assignment
    help with my assignment
    Online Assignment Help

    ReplyDelete
  46. Excellent Blog! I would like to thank you for the efforts you have made in writing this post. I am hoping for the same best work from you in the future as well. I wanted to thank you for this websites! Thanks for sharing.
    probability assignment help
    assignment writing services
    management assignment help
    nursing assignment help

    ReplyDelete
  47. Although the downloading process of the HP printer assistant is easy, yet most of the users are having difficulties to find that. If you are not able to find the software or unable to proceed towards the HP printer assistant download, do not get annoyed, our expert team would help you out from that easily.

    ReplyDelete
  48. Nice blog and absolutely outstanding. You can do something much better but i still say this perfect.Keep trying for the best
    System Restore error 0x800700b7

    ReplyDelete
  49. Superb blog and really great topic you choose. You know the most amazing thing I like in your article is your choosing words. Many times I see lots of new words in your article. I am also here for my Garmin Express web page promotion. Garmin Express in an app, which is used to manage Garmin devices. So if you using Garmin product and your Garmin Express Not Working or product requires Garmin express update then contact us from the Garmin Express Updates team. And please visit the web page for more information. Download Garmin express

    ReplyDelete
  50. I enjoyed over read your blog post i hope that you will be post more informative articles. Waiting for next article. Get help for your printer issues viva certified expert technician of our company 123helpline which is certified by many customer at any time 24 x 7 or visit our website www 123 hp com setup.

    ReplyDelete
  51. Installing the Avast antivirus software is an easy one can do. Some users have been reported that they are failed to know how to install Avast antivirus on their own. If you are too stuck around to install the program, feel free to contact our Avast expert and get instant help.

    ReplyDelete
  52. Norton security programs are easy to download and install. Still, most of the new users are not able to proceed through the Norton.com/setup download in order to get the program. If you are possibly facing some sort of issue while downloading the program, you must get connected with our technical team.

    ReplyDelete
  53. I must say you have written very nice article. Thanks for sharing it. The way you have described everything is phenomenal. You can follow us by visit our Web page HP Printer Wrieless Setup or Call our Toll-Free Number at anytime 24 x 7.

    ReplyDelete
  54. The printer goes to mistake state when your framework gets new updates making the associated gadget stop the activity. You ought to apply some investigating steps to determine canon.com-ijsetup.com. In addition, you can connect with specialists to fix the issue right away.

    ReplyDelete
  55. Epson Error Code | How to Troubleshoot Error Code 0X97?
    Epson printer is one such printing device that allows users to have a wide range of benefits. Along with this, the printer is also associated with several error code issues. If you are too getting one of the errors – Epson error code 0x97, you must get connected with our Epson printer experts.

    ReplyDelete
  56. How Do I Troubleshoot Gmail Not Receiving Emails 2020?
    The webmail – Gmail has been known for its quick sending and receiving emails. Yet, in some instances, users have been facing the issue of Gmail not receiving emails 2020. If you are not able to fix this sort of issue on your own, you must try contacting our Gmail technical executive right away.

    ReplyDelete
  57. If HP Envy 5540 Printer Offline Issues is one of the common problem for new printer or in old one. Hi Guys if you are looking for help regarding the HP Printer related any issues such as setup your printer or printer offline etc visit our Official website. Call our toll-free number we are available 24 x 7. Thank you for the post it was really good keep it up.

    ReplyDelete
  58. Yahoo mail is one of the well-known email service provided by Yahoo individuals to share any sort of message instantly with other users. However, supposedly, some of its users are facing the issue of Yahoo mail not responding properly in receiving emails. If you are facing such a similar issue in your mail account, you are just a call away to get the technical help to resolve your issue.

    ReplyDelete
  59. In the first place, access the Roku channel store and search for the PBS Kids channel. Click the “Add channel” tab and include the platform to the account. After this, open a web browser on your PC and type pbskids.org/activate roku. Tap the enter button and you’ll be redirected to the page. In this webpage, you have to paste and submit the code. This thereby activates the platform in the process. In case of any queries related to the PBS kids on roku, call the support team at +1-844-879-5200and get the issues fixed.

    ReplyDelete
  60. Hey buddy, I must say you have written very nice article. Thanks for sharing it. The way you have described everything is phenomenal. You can follow us by visit our Web page Step to Install Canon Pixma ip2820 Printer

    ReplyDelete
  61. Yahoo mail users would look over the email service providers for their own sake of sending and
    receiving emails. However, sometimes the user faces the issue of Yahoo mail not responding
    email perfectly. If you are the one facing the same, you can reach executives for assistance.

    ReplyDelete
  62. NobleQQ - Situs Poker Online Server GLX Terbaik dan Terpercaya di Indonesia Dijamin Aman dan Terpercaya
    CS 24/7 Online WA +855 1090 3838

    Promo 20rb GRATIS yuk buruan hanya di AGEN POKER ONLINE new member diberi 20k skuyy wa :+85510903838

    ReplyDelete
  63. To be honest I found very helpful information your blog thanks for providing us such blog cricket game

    ReplyDelete
  64. To be honest I found very helpful information your blog thanks for providing us such blog how to use grenades

    ReplyDelete
  65. To be honest I found very helpful information your blog thanks for providing us such blog Cricket Prediction

    ReplyDelete
  66. Thanks for sharing such a great information.. It really helpful to me.. Cyberpunk 2077 jacket I always search to read the quality content and finally i found this in you post. keep it up!

    ReplyDelete
  67. Have you as of late overhauled your Windows to Windows 10 and discovering challenges in printing with HP Laserjet 1200 Printer? In the event that indeed, at that point you have to Install hp laserjet 1200 manual On Windows10. To know how peruse the blog!

    ReplyDelete
  68. Get the best and cost-effective website during the covid-19 crisis at OGEN Infosystem. Thanks for sharing this valuable information with us.
    Website Designing Company

    ReplyDelete
  69. Epson printers are well known for their most user-friendly features. But when it comes to diagnosing a problem in the Epson printer, many would fail to proceed upon Epson printer troubleshooting features. If you are failed to troubleshoot your Epson printer to diagnose a problem, just get connected with our tech specialists right away.

    ReplyDelete
  70. contact QuickBooks Customer Service Number 1-833-325-0220 to overcome any kind of technical issues. Problem in QuickBooks is a matter of concern. The problem can be resolved efficiently with the support offered by our QuickBooks experts. Read More: https://tinyurl.com/ycms35td

    ReplyDelete
  71. We provide a detailed update regarding cricket matches on a daily basis. We offer you the best India fantasy information and cricket news and player. So that you Get Best today match prediction and increase your chances of a win. 2023 ODI World Cup

    ReplyDelete
  72. It’s really a cool and useful piece of information. I’m glad that you shared this useful information with us. Please keep us up to date like this. Thanks for sharing.epson printer troubleshooting error code 0x88

    ReplyDelete

  73. I am glad to investigate such an incredible substance. A dedication of appreciation is all together for sharing such a splendid bit of information with us. Keep sharing.Epson Error Code 0x83

    ReplyDelete
  74. If you are looking for a phone number to contact cash app support center, the you have landed on the right page. In the article below, we will tell you about Cash App and how to get support services for your accounts

    https://cashapphelp.online/
    https://cashapphelp.online/
    https://cashapphelp.online/
    https://cashapphelp.online/
    https://cashapphelp.online/
    https://cashapphelp.online/

    ReplyDelete
  75. How to Recover Forgot Yahoo ID & Password
    Basically, there are three different ways through which Yahoo account recovery can be done on an instant basis. The ways are none other than through phone number verification, security question answer mode, and alternate email authentication. In our blog we will guide you how to recover yahoo account works and how to easily recover a deleted account.

    ReplyDelete
  76. Contact Yahoo customer Support

    Whenever difficulties arise with your Yahoo email account, you will occasionally need to call Yahoo support to solve these difficulties. Yahoo Customer Service will resolve email account problems, problems, and questions with the billing of Yahoo’s small business website. There are some numbers that you can call to reach Yahoo customer service. Read the blog for more info.

    ReplyDelete

  77. hulu Customer Service team is available 24*7. You can get connected with.hulu.com/activate

    ReplyDelete
  78. Pogo Support is free to help you with issues about Pogo Games glitches. In this case, a Pogo Support live representative will assist you to locate the appropriate resource or team to solve your questions.

    ReplyDelete
  79. Epson Printer Error Code W-12 is one of the most widely used printing devices used by people across the world due to its advanced printing functionality.
    https://www.epsonsupports247.com/how-to-fix-epson-printer-error-code-w-12/

    ReplyDelete
  80. Reset the Chip on an Epson 69 Printer Cartridge on a printer cartridge is used to monitor the reserves and levels of the current toner cartridge.
    https://www.epsonsupports247.com/how-to-reset-the-chip-on-an-epson-69-printer-cartridge/

    ReplyDelete
  81. Great information! I thankful to the author of this blog who sharing such useful information, I also subscribe to your blog for all future posts. I have also shared some useful links.HP printer not printing

    ReplyDelete
  82. I am very glad to read such high-quality content. Thanks for sharing such a nice piece of information with us. Keep sharing…roku.com link

    ReplyDelete
  83. Epson Laser Printer Technical Support Number for immediate and instant support delivers better results. The toner-based printer is the most used device with the peripheral of the system that uses a laser drum
    https://www.epsonsupports247.com/epson-laser-printer-setup/

    ReplyDelete
  84. Much obliged for sharing the data yet you need more data to refresh your route framework.hp printer in error state
    hp printer install wizard
    hp support assistant download

    ReplyDelete