Alternative To Using A Multi Dimensional Array With A Huge Data Set For Front End Development?

Dev Zone Web Development
, ,
1

Intro

Okay, so I have a basic problem, I’ve been given a job to do, however, it’s 110% front end, I have 0 access to any back end tech, no back end scripting, no SQL, nout… This job is actually horrible in one sense, with this specific problem, I’m working with a huge data set, I mean this data set is so unrealistically huge that the multi dimensional array I’ve written is actually over 1,000 lines long, and it’s just filled with content that’s meant to APPEAR like it’s dynamic content.

Don’t ask as to why I have 0 back end access, I’ve brought it up myself, and I found very quickly that I was wasting my breath. * headache * … So I’ve had to mimic dynamic content, using JS alone.

Source Code - The Structure Of The Array

An example of what my array looks like would be something like this:

var array;

....

['String', float, float, 'String', 'String', 'String',
	[
		[
			'String||',
			''
		],
		[
			'String||',
			''
		]
	],
    'String'
], ......

Now the multi dimensional array itself runs more than fast enough, I won’t lie, I assumed due to the sheer size of it that it may perform a little slow, but I’ve been having no issues in relation to performance on any web browser. I mean I know I could’ve used objects or whatever, but it turns out that this is more than fast enough as it is, surprisingly, and there’s so much data, I’m not gonna go back and change all entries for syntax preferences. I will openly admit, I would have done this differently if I had known that I was going to be handling this much data, when I first started this task, I had 3 entries of data, so as you can imagine, the array was nice and small, and at a very exponential rate, the size of the data set grew.

Explenation

I hope it’s not just me that finds this disgusting. I think this is terrible, I mean I come from a full stack background, so doing EVERYTHING front end just seems so wrong. I personally think I need to find a more elegant solution, while there’s 0 performance issues, and there appears to be 0 bugs with the code I’ve written, it’s so ugly and annoying to maintain.

While it’s not of much importance, if you must know the purpose of this application, it’s basically just adding a TONNE of elements to a static page, images mostly, then the nested arrays are basically the blocks of content, one of the nested arrays goes into some DOM element, the other goes into a completely different DOM element, hence why they’re split up.

It almost looks like you’re getting dynamic content through ajax, only obviously you’re not.

And seriously… I cannot stress how shocked I am that there is NO performance hit with this array, I mean even on mobile devices, if I wasn’t the one building this page, I would’ve assumed that data was all set and brought in via the back end.

Finally

Does anyone have any recommendations on how to make this code more maintainable & easier to read?
This is surprisingly very fast, I mean for each element of this array, it dumps the data into an element, and that element is basically meant to act like an object then. I would use objects, but I may as well do it this way because these elements only gather the data when you click on this element.

For an example of what I mean, here’s a fiddle: https://jsfiddle.net/xaunr1ry/

2

I find your assumption strange. Why would an actual request to the server, an operation that includes several additional steps compared to having the data locally, be ever faster?
If it is about the memory usage by the browser or initial data transfer, then OK. I do not know the data set size.

As for arrays - I assume you are wondering why brute-force searches are so fast?
If not then, array is the fastest data structure you can have, assuming sequential or random access. Less so for searches - unless you have an order and do binary search. But even hash map is a structure that in the end is an set of organized arrays.

As for brute-force searches. In general, if you do not need to do thousands of queries per millisecond, the user will not see much difference. Mainly because your code preparing the data is only part of the time spent on the whole process of refreshing the page content (you already mentioned that the usually most consuming part - communication with remote server - is out of the picture)

And also, I do not know JS well, but I think any “multi dimensional array” is just array of arrays of arrays - so the memory usage is based on actual data not just multiplication of dimensions.

As for data structures - you already mentioned a good solution: Objects.
You could change arrays into the map of maps (etc. ). But better yet, wrapping each substructure into object/class in general will provide context in form of the method name (DSL - domain specific language).
So instead of a[1][3][5] or map.get(1).get(3).get(5) you can have:
cache.getMenuCategory(1).getMenuItem(3).getLabel()

As for dynamic content:
You can look into frameworks like Angular (Angular JS).
Or ditch the JS altogether and look for Dart and Angular for Dart.

1 Like
3

Sounds like a Polymer or Vue(using Vuex) problem. They are meant to show content based on dynamic data, though will require quite a lot of time investment.

4

Yes, that’s my exact thought, the size of the data is ridiculous, I mean in the ideal world it would be brought in from a database, so far the data takes up 1,000 lines, but that’s only a fractional amount, there’s still a LOT more data to be entered.

Isn’t that pretty much all a multi dimensional array is?.. I mean to my knowledge, that’s pretty much all they are, sure you could do a multi dimensional array where an array contains an array, but the nested array will only ever have x elements, which kinda defeats the purpose in my eyes, in such a case, it should just become an array of objects?

FYI. You say you don’t know JS very well, ‘formal’ classes don’t really exist in JS quite yet, I mean they do, but they’re buggy on older browsers, the usual way of making an object in JS is kinda like creating a struct, but like a class, you can have functions associated with the instance of that class.

I would do that, it would even give me an excuse to start using Angular with my commercial work, but sadly… I can’t use Angular because no other developer uses/knows angular, which is surprising in this day and age… Obviously you CAN have objects in JS, if anything, with JS, it actually treats the HTML elements as objects, like in the example I showed, you can actually use the keyword ‘this’. But as for writing classes as you would in Java, or C++ or whatever language… JS can do it, it’s just not a good idea if you wanna support older browsers…

Doing it the old school way where you do it something like this:

var human = {
      name:'',
     	age:0,
      weight:0
	};

	var x = human;
	x.name = 'Jack';
	x.age = 20;
	x.weight = 150;
	console.log(x.name + ' is ' + x.age + ' years old and weighs ' + x.weight + 'lbs');

I mean it works, but compared to Java or C++ it feels like JS has only partially implemented OOP. To me personally at least… I’m sure many may disagree, maybe many will also agree, but that is honestly just my personal opinion, and that’s why I’m glad I learned Java, it emphasised OOP, big time.

Tell me about it! :joy:

I’ve already spent a week on it, and that’s just been adding data, I’m half tempted to make an ‘admin page’ that’ll dump out data that’s entered into a nice UI into an array and it dumps that array into a new file. I mean doing ctrl + f for some unique ID or something, it gets old quickly… I’ve spent a lot of time squinting at this specific array! :joy:

Edit

Basically, I’m using the array to kinda create objects, emphasis on the word kinda… And for each object, there’s an event listener, when this event is fired, that is when the content ‘dynamically’ changes.

And as for @jak_ub, I’m sorry if I wasn’t clear, I don’t and was never under the impression that using some back end interaction would speed it up, quite the opposite, I just feel that it would make it a lot easier to maintain, that could be down to the fact that I’m originally a full stack developer, who knows? I’m used to building an admin application to add the data, a back end script to do the heavy lifting, then a front end UI just to do funky things to make it look pretty and add animations and all that jazz.

5

It depends on the language. There are languages that literally allocate continuous memory block of size xyz*n of pointers.

Anyway, based on the task limitations you have (not sure how other tasks look like - although I remember that this is not the first strictly JS related issue you resolve), I’m maybe talking out of my ass right now, but:
maybe its time to look for better employment.

6

I agree… :joy: … Strongly… I mean this company, nice guys, but the way they do somethings, it genuinely makes you wonder why on earth any one would do it like that! :stuck_out_tongue:

That’s one good thing though, this job has SERIOUSLY improved my JS skills, I look back now and laugh… :joy:

7

Update

Considering how much data I ended up needing to include for each element of this array, I just converted it into an array of JSON structures, with arrays, and what not.

I’ve decided to just say f*!k it and use JSON because of how messy things were getting, it makes it easier to maintain and read. I.E. Rather than doing something like:

var title = array[i][0], someOtherVar = array[i][1], someNestedArray = [i][2];

It can now just be

var obj = jsonArray[i];
obj.title;
obj.someOhterVar;
obj.someNestedArray;

… So on and so forth… Personally I think that’s better form a maintainability perspective, performance, there appears to be no noticeable difference.